Senior Data Engineer

Job Description

Responsibilities:

• Review mapping definition with data analysts
• Contribute with other data scientists on mapping definition for transformation pipelines
• Contribute to mapping function development and unit testing of these functions
• Contribute to mapping configurations
• Define the design of quality control pipelines, ensuring mapping accuracy and alignment with analytical expectations
• Review and maintain the quality control pipelines with the Data Analyst, Data Domain Lead and Data Governance Lead
• Implement or configure the appropriate tools for the approved quality control pipelines
• Provide the guidelines for introducing Machine Learning and LLM in the transformation pipelines

What we expect:

• 10+ years of Data Engineering experience utilising designated technologies and tools.
• Profound knowledge and hands-on experience with Python (Pandas, boto3), SQL, Spark, Databricks, DWH
• Experience in implementing ML and LLM in production data pipelines
• Expertise with AWS services essential for Data Engineering: AWS Glue, AWS Lambda, S3, RDS, Redshift, Athena, SQS, SNS, etc.
• Ability to write robust code with Python.
• Deep understanding of pub-sub architecture
• Strong upper-intermediate or higher (B2+) proficiency in English
• Solid analytical and problem-solving skills

Nice to have:

• Experience with AWS Dynamo DB, Aurora (including serverless nuances)
• Familiarity with AWS Networking
• Familiarity with Azure
• Experience with Snowflake
• Familiarity with SODA

Are you interested in this position?

Apply by clicking on the “Apply Now” button below!

#GraphicDesignJobsOnline

#WebDesignRemoteJobs #FreelanceGraphicDesigner #WorkFromHomeDesignJobs #OnlineWebDesignWork #RemoteDesignOpportunities #HireGraphicDesigners #DigitalDesignCareers# Dynamicbrand guru