Senior Data Engineer

Equal Opportunity Employer

All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, protected veteran status, or disability status. EEO/AA/M/F/Disabled/Vets

Job Description :

Senior Data Engineer

One of the most innovative and high-profile teams in News Corp is looking for a seasoned data engineer to help accelerate its vision. The role will involve combining all of News Corp’s data across categories, countries and organizations into a singular view of a customer enabling engagement, measurement and targeting. The HQData Senior Data Engineer will partner closely with product, data & engineering teams to help make this data available and accessible to our businesses, teams and partners in order to drive revenue and improve the customer experience.

Responsibilities

• Design, implement and maintain high performance big data infrastructure/systems & big data processing pipelines scaling to billions of structured and unstructured events daily
• Design, implement and maintain deep integration with up-stream systems
• Monitor performance of the data platform and optimize as needed
• Monitor and provide transparency into data quality across systems (accuracy, consistency, completeness, etc)
• Support product with the overall roadmap and ensure updates to senior leadership are 100% technically correct.
• Data analysis, understanding of business requirements and translation into logical pipelines & processes
• Conduct timely and effective research in response to specific requests (e.g. data collection, summarization, analysis, and synthesis of relevant data and information)
• Evaluate and prototype new technologies in the area of data processing
• Think quickly, communicate clearly and work collaboratively with product, data, engineering, QA and operations teams
• High energy level, strong team player and good work ethic
  
  

Technologies we use

• Google Cloud Platform (Storage, Dataflow, BigQuery, DataStudio, DataTransfer, Composer)
• Python, R, Spark (PySpark)
• Kubernetes
• Airflow
• Git for source code management
• Jira
  
  

Qualifications

• BS in Computer Science or other technical discipline
• 5+ years of experience in designing and developing big data processing systems using distributed computing
• 5+ years in Python Programming
• 5+ years experience in SQL
• 4+ years experience in Apache Airflow
• Working with REST APIs
• 5+ years of Hands on experience in Google Cloud Platform
• Expert knowledge in optimizing complex SQL queries
• Experience with job orchestration tools
• An affinity for automation
• Familiarity with networking and network application programming, including HTTP/HTTPS, JSON, and REST APIs
• Experience with at least one object oriented language (ex: Java)
• Strong OO design, data structure, and algorithm design skills
  
  

This job is posted with NTS Technology Services Pvt. Ltd.

Job Category: