Data Engineer- Gen AI - Senior Associate

Line of Service

Advisory

Industry/Sector

Not Applicable

Specialism

Data, Analytics & AI

Management Level

Senior Associate

Job Description & Summary

A career within Data and Analytics services will provide you with the opportunity to help organisations uncover enterprise insights and drive business results using smarter data analytics. We focus on a collection of organisational technology capabilities, including business intelligence, data management, and data assurance that help our clients drive innovation, growth, and change within their organisations in order to keep up with the changing nature of customers and technology. We make impactful decisions by mixing mind and machine to leverage data, understand and navigate risk, and help our clients gain a competitive edge.

As part of our Analytics and Insights Consumption team, you’ll analyze data to drive useful insights for clients to address core business issues or to drive strategic outcomes. You'll use visualization, statistical and analytics models, AI/ML techniques, Modelops and other techniques to develop these insights.

Job Description (Senior Associate):

- Design, develop, and maintain data pipelines and ETL processes for GenAI projects.

- Collaborate with data scientists and software engineers to implement machine learning models and algorithms.

- Optimize data infrastructure and storage solutions to ensure efficient and scalable data processing.

- Implement event-driven architectures to enable real-time data processing and analysis.

- Utilize containerization technologies like Kubernetes and Docker for efficient deployment and scalability.

- Develop and maintain data lakes for storing and managing large volumes of structured and unstructured data.

- Implement and integrate LLM frameworks (Langchain, Semantic Kernel) for advanced language processing and analysis.

- Collaborate with cross-functional teams to design and implement solution architectures for GenAI projects.

- Utilize cloud computing platforms such as Azure or AWS for data processing, storage, and deployment.

- Monitor and troubleshoot data pipelines and systems to ensure smooth and uninterrupted data flow.

- Stay up-to-date with the latest advancements in GenAI technologies and recommend innovative solutions to enhance data engineering processes.

- Collaborate with cross-functional teams to understand business requirements and translate them into technical solutions.

- Document data engineering processes, methodologies, and best practices.

- Maintain solution architecture certificates and stay current with industry best practices.

Preferred Skills:

• Gen AI Frameworks:  Experience with LLM frameworks or tools for interacting with LLMs such as LangChain, Semantic Kernel, LlamaIndex

• Data Pipelines: Experience in setting up data pipelines for model training and real-time inference.

Education (if blank, degree and/or field of study not specified)

Degrees/Field of Study required:

Degrees/Field of Study preferred:

Certifications (if blank, certifications not specified)

Required Skills

Optional Skills

Desired Languages (If blank, desired languages not specified)

Travel Requirements

Not Specified

Available for Work Visa Sponsorship?

No

Government Clearance Required?

No

Job Posting End Date