Lead Data Engineer – Python/Java

Job Description

As a Lead Data Engineer, you will serve as a strategic technical leader, architecting, scaling, and optimizing enterprise-wide data platforms. You will drive innovation, set technical direction, and mentor a high-performing team to deliver robust, secure, and high-quality data solutions that power business-critical analytics and applications across the organization.

Key Responsibilities :

Technical Leadership & Strategy :

– Define and drive the data engineering vision, strategy, and roadmap in alignment with business goals.

– Architect and oversee the design of complex, distributed data systems, ensuring scalability, reliability, and security.

– Establish and enforce technical standards, best practices, and governance for data engineering across teams.

– Advise on technical specifications and collaborate with product owners, business partners, and cybersecurity teams to clarify requirements and dependencies.

Team Leadership & Mentoring :

– Lead, mentor, and grow a team of data engineers, fostering a culture of innovation, accountability, and continuous improvement.

– Conduct regular code and design reviews, promoting clean, efficient, and maintainable code.

– Champion professional development and knowledge sharing within the team.

Project & Stakeholder Management :

– Own end-to-end delivery of large-scale data engineering projects, including prioritization, planning, and stakeholder management.

– Track external commitments and ensure timely delivery of features and enhancements.

– Identify and drive opportunities for process improvements and automation.

Technical Excellence & Innovation :

– Resolve complex technical issues and make impactful decisions that shape the future of data engineering at the enterprise level.

– Lead the evaluation and adoption of emerging technologies (e.g., cloud platforms, streaming, ML/AI integration).

– Ensure production systems are robust, performant, and meet SLAs.

– Proactively identify and resolve bottlenecks.

Cross-Functional Collaboration :

– Work closely with senior engineers, architects, and product managers to develop innovative solutions for new product development.

– Collaborate with business and analytics teams to deliver data products that unlock business value.

This is a hybrid position. Expectation of days in office will be confirmed by your hiring manager.

Qualifications :

Basic Qualifications :

– 14 or more years of relevant work experience with a bachelors degree or at least 5 years of experience with an Advanced Degree (e.g. Masters, MBA, JD, MD) or 2 years of work experience with a PhD, OR 14+ years of relevant work experience.

Preferred Qualifications :

– Proven experience architecting and leading enterprise-scale data engineering initiatives (Hadoop, Hive, Spark, Kafka, etc.).

– Strong programming proficiency in Python, Java, or Scala.

– Production-level experience architecting Hadoop, Spark, and Databricks data pipelines.

– Hands-on experience with AWS or Azure (Glue, Synapse, Redshift, Delta Lake, S3).

– Familiarity with Kafka, Airflow, Kubernetes, and containerized data services.

– Deep expertise in cloud platforms (AWS, Azure, GCP) and associated services (EC2, S3, SageMaker, etc.).

– Advanced proficiency in programming languages (Scala, Python, Java) and scripting (Unix/Shell, Python).

– Understanding of RAG pipelines, vector databases, and AI data flows.

– Experience designing data pipelines and APIs compatible with Model Context Protocol (MCP)-based agent frameworks, enabling seamless integration between AI agents, data services, and enterprise API

– Strong background in data modeling, ETL/ELT, and optimization of big data pipelines.

– Experience with CI/CD, automated testing, and DevOps tools (Jenkins, Artifactory, Git, Selenium, Chef).

– Familiarity with RDBMS (MS SQL, DB2, Oracle) and visualization tools (Tableau, Power BI).

– Demonstrated ability to lead teams, influence stakeholders, and deliver results in a fast-paced, global environment.

– Excellent communication, leadership, and organizational skills.

Are you interested in this position?

Apply by clicking on the “Apply Now” button below!

#GraphicDesignJobsOnline

#WebDesignRemoteJobs #FreelanceGraphicDesigner #WorkFromHomeDesignJobs #OnlineWebDesignWork #RemoteDesignOpportunities #HireGraphicDesigners #DigitalDesignCareers# Dynamicbrand guru