Design and develop scalable and reliable data pipelines and infrastructure

Optimize data processing pipelines for performance and scalability

Design and implement data storage solutions

Ensure data quality and integrity by implementing data validation and testing procedures

Collaborate with cross-functional teams to gather data from various sources and transform it for reporting needs

Develop and maintain documentation of data pipelines and infrastructure

Provide technical guidance and mentorship to junior team members

A minimum of 3 years of experience in Data Software Engineering, demonstrating your expertise in data processing and storage solutions

Strong experience with AWS services, including EC2, S3, Lambda and RDS

Proficiency in Apache Spark, PySpark and Apache Airflow, enabling you to develop and maintain data processing pipelines

Experience with Python and Snowflake, allowing you to store and manage large amounts of data

Strong experience with Kafka, enabling you to build real-time data processing pipelines

Fluent spoken and written English at an Upper-Intermediate level or higher, enabling effective communication

Experience with data visualization tools, such as Tableau or Power BI

Experience with data governance and security best practices