Develop and maintain data pipelines and analytics solutions

Optimize Spark jobs for performance and scalability, leveraging AWS services

Contribute to the development of data architecture and data modeling

Ensure data accuracy and quality control through unit tests and columnar-based file types

Participate in code reviews and contribute to the development of best practices for data engineering

Collaborate with cross-functional teams to understand business requirements and data needs, contributing to the development of data-driven solutions

Stay up-to-date with the latest data engineering technologies and trends

A minimum of 3 years of experience in Data Software Engineering, with a strong understanding of the Data domain

Experience with Apache Spark and PySpark, as well as SQL and Big Data technologies

Proficiency in Amazon Web Services (AWS), including AWS CLI, S3, Glue, Lambda, Athena, and Step Functions

Familiarity with columnar-based file types and unit testing

Excellent communication skills and strong critical thinking capabilities to effectively convey feedback and insights

Fluent spoken and written English at an Upper-Intermediate level or higher, enabling effective communication

Experience with CI/CD tools such as Atlassian Bitbucket, AWS CodeBuild, and Terraform