Data Engineer – Python Programming

Job Description

Core Responsibilities :

– CPG Data Harmonization : Designing pipelines to ingest, harmonize and curate disparate data sources, including ERP (eg., SAP), PLM (eg., Laboratory Information Management System), Point of Sale (POS) data from global retailers (e.g., Walmart, Costco, Carrefour, Tesco, etc.), and internal WMS/TMS systems into a unified Delta Lake.

– Lakehouse Implementation : Developing a Medallion Architecture (Bronze, Silver, Gold) in

Databricks to ensure high-quality, trusted aggregated data model and curated data products.

– Supply Chain Orchestration : Using Databricks Workflows or Azure Data Factory to automate complex ETL/ELT processes that power demand forecasting and inventory replenishment models.

– Operational Monitoring : Implementing real-time streaming (Spark Structured Streaming) to provide immediate visibility into supply chain disruptions, such as logistics bottlenecks or stockouts.

– Governance & Security : Managing fine-grained access control and data lineage for multi-brand or regional datasets using Unity Catalog to ensure compliance with global data privacy standards.
Required Technical Skills :

– Education : Bachelor’s degree in engineering, computer science, or related field.

– Programming : 5 – 8 years of total work experience, with at least 3 years of experience in advanced SQL, PySpark or Python.

– Core Platform : Good grasp of Databricks (Delta Lake, DLT, Workflows) and Azure/Cloud services (ADLS Gen2, Key Vault).

– Data Lineage : Experience with leveraging Unity Catalog for ensuring lineage from raw to curated data assets.

– DevOps/DataOps : Proficiency in Git, CI/CD (Azure DevOps/GitHub Actions), and Infrastructure as Code (Terraform) for stable, repeatable deployments.

– Agile Development : Experience working in an collaborative agile environment.

– Cloud Architecture : Understanding of cloud architecture principles and best practices

– Experience in the design and build of end-to-end solutions that meet business requirements

and adhere to scalability, reliability, and security standards

Desired Certifications :

– Microsoft Certified : Azure Fundamentals

– Microsoft Certified : Azure Data Engineer Associate

– Databricks Certified Data Engineer Associate

Are you interested in this position?

Apply by clicking on the “Apply Now” button below!

#GraphicDesignJobsOnline

#WebDesignRemoteJobs #FreelanceGraphicDesigner #WorkFromHomeDesignJobs #OnlineWebDesignWork #RemoteDesignOpportunities #HireGraphicDesigners #DigitalDesignCareers# Dynamicbrand guru