Data Quality Engineer

Job Description

About the Role :

We are looking for an experienced QA & Data Validation Engineer with 5 – 6 years of hands – on experience in data quality assurance, solution analysis, data validation, SQL, Python, PySpark, Azure Data Factory, Azure Synapse Analytics, and Power BI validation.

The ideal candidate will be responsible for validating large – scale data pipelines, performing source – to – target reconciliation, analyzing business rules, investigating data defects, and ensuring the accuracy, completeness, and consistency of data across source, staging, intermediate, and target systems.

The role requires strong analytical and problem – solving skills along with the ability to work closely with development, data engineering, business, and other stakeholders in an Agile delivery environment.

You will also contribute to the design, development, and maintenance of automated validation frameworks and utilities using Python, SQL, PySpark, Azure Data Factory, and Azure Synapse.

Key Responsibilities :

– Analyze business and technical requirements to understand data processing and validation needs.

– Participate in requirement analysis sessions and clarify functional and technical requirements with stakeholders.

– Review solution designs, data flows, mapping documents, interface specifications, and business rules.

– Validate that implemented solutions meet defined business and technical requirements.

– Identify gaps, inconsistencies, ambiguities, and potential data quality issues during requirement and solution analysis.

– Translate business requirements into detailed test scenarios, test cases, and validation conditions.

– Perform end – to – end validation of data processing workflows.

– Ensure data is accurately processed from source systems through intermediate layers to final outputs.

– Validate business rules and transformation logic implemented within data pipelines.

Test Planning & Execution :

– Prepare comprehensive test strategies, test plans, test scenarios, and test cases for data – intensive applications.

– Execute functional, integration, regression, system, and data validation testing.

– Perform positive and negative testing for different data processing scenarios.

– Validate data pipelines across multiple environments, including staging, testing, and production.

– Identify test data requirements and prepare appropriate datasets for validation.

– Execute SQL queries to validate data processing and transformation results.

– Document test results, observations, defects, and validation evidence.

– Track testing progress and communicate status, risks, issues, and dependencies to stakeholders.

Data Validation & Reconciliation :

– Perform detailed source – to – target data validation and reconciliation.

– Validate source, intermediate, staging, and output datasets.

– Perform record count validation between source and target systems.

– Verify data completeness, consistency, accuracy, and integrity.

– Validate data transformations against defined business rules.

– Perform field – level and record – level comparisons.

– Validate data types, formats, precision, scale, and null handling.

– Verify schema structure, layout, column names, and column sequence.

– Validate mandatory and optional fields.

– Identify missing, duplicate, truncated, or incorrectly transformed records.

– Analyze invalid records, rejected records, and exception datasets.

– Verify exception and reject – handling mechanisms.

– Compare production and staging data to identify discrepancies.

– Perform reconciliation between files, databases, and reporting layers.

– Validate data across different processing stages and identify the root cause of discrepancies.

Are you interested in this position?
Apply by clicking on the “Apply Now” button below!
#GraphicDesignJobsOnlineAre you interested in this position?
Apply by clicking on the “Apply Now” button below!
#GraphicDesignJobsOnline
#WebDesignRemoteJobs
#FreelanceGraphicDesigner
#WorkFromHomeDesignJobs
#OnlineWebDesignWork
#RemoteDesignOpportunities
#HireGraphicDesigners
#DigitalDesignCareers