Apply Apply

About the role

Are you a Lead Data Engineer with a passion for big data? Do you keep up to date with cutting-edge technologies within D&A?

If so, then you have a fantastic opportunity to join a multi-disciplinary team of engineers, architects, designers, and strategists as we continue to grow our Data & Analytics practice across Europe.
We’re looking for a Lead Data Engineer with Azure, Databricks and PySpark to join our team in London. The ideal candidate will have a strong background in data engineering, extensive experience with Azure cloud services, and experience leading a technical team on the implementation.

Responsibilities

• Lead, mentor and manage a team of Azure data engineers
• Drive the team's technical execution
• Collaborate with cross-functional teams including data scientists, analysts and business stakeholders ensuring a quality single version of truth
• Passionate engineer, very keen on building end to end pipelines to support enterprise-wide analytics
• Design, develop and implement scalable and secure data lake solutions on Azure
• Ensure best practices in data engineering, data integration and ETL processes
• Prepared to code complicated aspects of our pipeline
• Ensure the ongoing maturity of our SVOT framework
• Monitor performance and scalability of the SVOT platform updating the framework and code to ensure the business has highly available accessible product

Requirements

• Minimum of 8 years of experience in data engineering
• At least 5 years of hands-on experience with Azure data services (Apache Spark, Azure Data Factory, Synapse Analytics, RDBMS experience (prefer SQL Server)
• Proven experience in leading and managing a team of data engineers
• Proficiency in programming languages specifically PySpark, Python (with Pandas if no PySpark), Continuous Integration (DevOps, PRs, Branching), T-SQL & SparkSQL
• Strong understanding of data modeling, ETL processes and data warehousing concepts
• Knowledge of CI/CD pipelines and version control (e.g., Git)
• Excellent problem-solving and analytical skills
• Strong communication and collaboration abilities
• Ability to manage multiple projects and meet deadlines
• Certifications in Azure (e.g., Microsoft Certified: Azure Data Engineer Associate, Azure Solutions Architect)

Nice to Have

• Hands-on experience with Scala for Apache Spark
• Knowledge or experience working with other Clouds such as AWS or GCP

Our Benefits Include

• A competitive group pension plan and protection benefits including life assurance, income protection and critical illness cover
• Private medical insurance and dental care
• Cyclescheme, Techscheme and season ticket loans
• Employee assistance program
• Great learning and development opportunities, including in-house professional training, career advisory and coaching, sponsored professional certifications, well-being programs, LinkedIn Learning Solutions and much more
• EPAM Employee Stock Purchase Plan (ESPP)
• Various perks such as gym discounts, free Wednesday lunch in-office, on-site massages and regular social events
• Certain benefits and perks may be subject to eligibility requirements and may be available only after you have passed your probationary period

About EPAM

• EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential