We are seeking a Lead Data Software Engineer to take ownership of designing and implementing advanced data engineering solutions.

Our client specializes in delivering cutting-edge property insights and data analytics, enabling clients in real estate, financial services and insurance to make informed decisions backed by comprehensive data. This role requires both hands-on expertise and the ability to guide a team of engineers. You will work in a dynamic and challenging environment, collaborating with a transitioning client team to ensure seamless knowledge transfer and successful project delivery.

The ideal candidate will have extensive experience with Python, Spark and cloud platforms and strong leadership and problem-solving skills.

Responsibilities

• Lead the design and development of scalable data pipelines and architectures using Python, Spark, Airflow and Kafka
• Architect and implement robust data storage and processing solutions leveraging AWS (DynamoDB, Glue, S3, Lambda, Fargate) or GCP and Snowflake
• Provide technical leadership and mentorship to a team of engineers
• Drive knowledge transfer from the client’s onshore team and ensure continuity in project delivery
• Collaborate with stakeholders to define technical requirements and deliver effective data solutions
• Oversee the implementation of containerized solutions using Docker, with a focus on scalability and reliability
• Ensure adherence to best practices in code quality, performance, and security
• Work within an Agile framework (e.g., Scrum, Kanban) and lead the team in adopting DevOps practices such as CI/CD pipelines and automated testing
• Use GitFlow or similar version control workflows for development and deployment

Requirements

• Extensive experience with Python, Spark and SQL in large-scale data environments
• Proven ability to lead technical teams and deliver complex projects
• Deep understanding of cloud-based ecosystems and distributed data systems
• Strong interpersonal and stakeholder management skills to address challenges in a transitioning client team
• Exposure to advanced technologies, such as serverless architecture and data lake design
• Highlight time zone flexibility to collaborate effectively with an Australian client
• Understanding and proficiency in general big data concepts

We offer

• By choosing EPAM, you're getting a job at one of the most loved workplaces according to Newsweek 2021 & 2022&2023
• Employee ideas are the main driver of our business. We have a very supportive environment where your voice matters
• You will be challenged while working side-by-side with the best talent globally. We work with top-notch technologies, constantly seeking new industry trends and best practices
• We offer a transparent career path and an individual roadmap to engineer your future & accelerate your journey
• At EPAM, you can find vast opportunities for self-development: online courses and libraries, mentoring programs, partial grants of certification, and experience exchange with colleagues around the world. You will learn, contribute, and grow with us

Life at EPAM

• EPAM is a leader in the fastest-growing segment (product development/digital platform engineering) of the IT industry. We acquired Just-BI in 2021 to reinforce our leading position as a global Business Intelligence services provider and have been growing rapidly. With a talented multinational team, we provide data and analytics expertise
• We are currently involved in end-to-end BI design and implementation projects in major national and international companies. We are proud of our entrepreneurial start-up culture and are focused on investing in people by creating continuous learning and development opportunities for our employees who deliver engineering excellence for our clients