Data Engineer

Job Description

Responsibilities :

– Identify downstream implications of data loads/migration (e. g., data quality, regulatory, etc. ).

– Implement data pipelines to automate the ingestion, transformation, and augmentation of data sources, and provide best practices for pipeline operations.

– Capability to work in a rapidly changing business environment and to enable simplified user access to massive data by building scalable data solutions.

Requirements :

– Must Have: GCP + Scala + Kafka or Pub/Sub.

– Have Implemented and Architected solutions on Google Cloud Platform using the components of GCP.

– Experience with Apache Beam/Google Dataflow/Apache Spark in creating end-to-end data pipelines.

– Experience in some of the following: Python, Hadoop, Spark, SQL, BigQuery, Bigtable, Cloud Storage, Datastore, Spanner, Cloud SQL, and Machine Learning.

– Experience programming in Java, Python, etc.

– Expertise in at least two of these technologies: Relational Databases, Analytical Databases, and NoSQL databases.

– Certification as a Google Professional Data Engineer/Solution Architect is a major Advantage.

– 8+ years’ experience in IT or professional services experience in IT delivery or large-scale IT analytics projects.

– Candidates must have expertise and knowledge of Google Cloud Platform; the other cloud platforms are nice to have.

– Expert knowledge in SQL development.

– Expertise in building data integration and preparation tools using cloud technologies (like Snaplogic, Google Dataflow, Cloud Dataprep, Python, etc. ).

– Experience with Apache Beam/Google Dataflow/Apache Spark in creating end-to-end data pipelines.

– Experience in some of the following: Python, Hadoop, Spark, SQL, BigQuery, Bigtable, Cloud Storage, Datastore, Spanner, Cloud SQL, and Machine Learning.

– Experience programming in Java, Python, etc.

– Advanced SQL writing and experience in data mining (SQL, ETL, data warehouse, etc. ) and using databases in a business environment with complex datasets.

Are you interested in this position?

Apply by clicking on the “Apply Now” button below!

#GraphicDesignJobsOnline

#WebDesignRemoteJobs #FreelanceGraphicDesigner #WorkFromHomeDesignJobs #OnlineWebDesignWork #RemoteDesignOpportunities #HireGraphicDesigners #DigitalDesignCareers# Dynamicbrand guru