Collaborate with the team on the migration of GCP BigQuery and DataFlow pipelines to Databricks hosted on AWS, ensuring a seamless transition

Design, develop, and optimize data pipelines using Python, Spark, and Databricks to meet business requirements and enhance data processing efficiency

Work closely with cross-functional teams to understand data requirements and contribute to the architecture and implementation of data solutions

Perform data modeling and transformation tasks to prepare data for analysis and reporting

Collaborate on code reviews, ensuring the quality and reliability of data pipelines

Troubleshoot and resolve issues related to data pipelines, ensuring smooth and consistent data flow

A minimum of 3 years of relevant experience as a Data Software Engineer, demonstrating your proficiency in developing and optimizing data solutions

Strong familiarity with AWS, leveraging its resources for hosting and managing data solutions

Proficiency in Python, utilizing its capabilities to build robust and efficient data pipelines

Experience with Apache Spark, enabling you to work with large-scale data processing and analysis

Expertise in Databricks, particularly in the context of migrating pipelines and working on AWS-hosted platforms

Solid command of SQL, allowing you to perform data transformations and analysis effectively

Fluent English communication skills at a B2+ level, facilitating effective collaboration with a diverse team

Familiarity with GCP, particularly experience with BigQuery and DataFlow

Knowledge of GCP BigQuery and DataFlow, aiding in the migration process and facilitating collaboration across platforms