Engineer

Job Description

Responsibilities

• Build and maintain scalable ETL/ELT pipelines and data models using Databricks and PySpark.
• Develop data transformations and processing workflows using Python and SQL.
• Work with Delta Lake and lakehouse architecture.
• Design and maintain data models and data marts for analytics.
• Optimize Spark jobs and SQL queries for performance and scalability.
• Implement data quality, validation, monitoring, and error handling.
• Work across cloud platforms (AWS, Azure, or GCP) depending on the client engagement.
• Use Databricks Workflows, Airflow, or similar tools for orchestration, depending on the client’s stack.
• Manage pipeline code through Git-based version control and CI/CD workflows.
• Collaborate with analytics, BI, and engineering teams and directly with international clients to deliver end-to-end data solutions.

Required Skills

Must-have

• 35 years of experience in Data Engineering.
• Strong hands-on experience with Databricks and Apache Spark.
• Proficiency in PySpark, Python, and SQL.
• Professional services client-facing experience and strong communication skills
• Experience with lakehouses
• Strong understanding of ETL/ELT pipelines and data modelling.
• Working knowledge of at least one major cloud platform (AWS, Azure, or GCP) comfort picking up a second is a plus given our multi-client environment.
• Experience with Airflow, Databricks Workflows, or similar orchestration tools.
• Strong problem-solving and analytical skills.
• Comfortable working directly with international (primarily US) clients and adapting to a partial evening-hours overlap.

Nice to have

• Experience with Unity Catalog or similar data governance frameworks.
• Familiarity with Git-based version control and CI/CD for data pipelines (e.g., Databricks Asset Bundles, Terraform).
• Exposure to Structured Streaming or real-time data pipelines.
• Databricks certification (Data Engineer Associate or Professional).

Are you interested in this position?
Apply by clicking on the “Apply Now” button below!
#GraphicDesignJobsOnline
#WebDesignRemoteJobs
#FreelanceGraphicDesigner
#WorkFromHomeDesignJobs
#OnlineWebDesignWork
#RemoteDesignOpportunities
#HireGraphicDesigners
#DigitalDesignCareers
# Dynamicbrand guru