Senior Automation QA (Big Data)

N-iX is an international software development service company that partners with businesses worldwide to help them create successful software products.
Our client is involved in a large-scale Data Transformation project, with a focus on solidifying the foundation of their data operations. They are aiming to ensure that data is accurate, consistent, and available at critical times to support their business needs.
The objective of this project is to enhance the QA processes through the implementation of automation within the Data landscape. This includes developing and maintaining regression testing suites, identifying tools to automate manual scenarios, and creating test scripts and automation plans to improve the overall efficiency and quality of data testing practices.

Responsibilities:

• Develop and automate processes for gathering expected results from data sources and comparing them with actual test results.
• Develop and maintain automated test scripts using Python, PySpark, and SQL for data validation.
• Source systems AX, CRM Dynamics, mediation, OCS
• Implement integration tests, and regression tests for data pipelines.
• Assist in the development and maintenance of smoke, performance, functional, and regression tests to ensure that code operates as intended.
• Collaborate with the team to understand how changes in the Data Layer affect test scripts and the automated testing environments.
• Own and enhance the test automation framework, ensuring appropriate automation is built where necessary.
• Write, monitor, execute, and evaluate application tests using industry-standard automated testing tools.
• Set up data, tools, and databases to facilitate the testing process.
• Foster and maintain constructive working relationships within the QA team and with other functional teams. 

Requirements:

• 4+ years of technical QA experience within Big Data Technologies.
• 2-3 years of experience working with automation frameworks.
• Proficiency in writing test code in Python, Pytest, or Robot.
• Strong SQL skills and experience with big data cloud platforms.
• Demonstrated experience in writing and maintaining automated tests for Big Data projects.
• Knowledge of and experience working with Apache Spark.
• Familiarity with BDD frameworks such as Cucumber or SpecFlow.
• Extensive knowledge and experience with Data Testing Strategies, including data validation, process validation, outcome validation, and code coverage.
• Experience in executing automated Big Data Testing tasks, including Pen Testing, Architecture Testing, Migration Testing, Performance Testing, Security Testing, and Visualization Testing.
• Expertise in automating the testing of various data sources (Relational, Flat Files, XML, NoSQL, Cloud, and Big Data).
• Hands-on experience with ETL Validator Testing Tools for automating ETL/ELT validation.
• Experience in test setup, software installations, and pipelines within CI/CD environments.
• Proficiency with monitoring tools like New Relic and Grafana.

Nice to Have:

• Advanced knowledge of public cloud platforms (Azure).
• Familiarity with database administration and optimization techniques.
• Experience with big data technologies (Spark).