We are seeking an experienced Lead Software Engineer (Big Data) to architect, develop, and lead scalable data solutions.

This role combines deep expertise in big data technologies with leadership skills to drive innovation and technical excellence in large-scale data projects. The successful candidate will collaborate with cross-functional teams, mentor engineers, and deliver solutions that meet both business and technical requirements in a cloud or hybrid environment.

Responsibilities

• Lead the design and development of scalable data pipelines and architectures using Python, Spark, Airflow, and Kafka
• Architect and implement robust data storage and processing solutions leveraging cloud platforms such as AWS (DynamoDB, Glue, S3, Lambda, Fargate), GCP, or Snowflake
• Provide technical leadership and mentorship to a team of engineers
• Drive knowledge transfer from the client’s onshore team, ensuring project continuity and high-quality delivery
• Collaborate with stakeholders to define technical requirements and deliver effective data solutions
• Oversee the implementation of containerized solutions using Docker, focusing on scalability and reliability
• Ensure adherence to best practices in code quality, performance, and security
• Work within Agile frameworks (e.g., Scrum, Kanban) and lead the adoption of DevOps practices such as CI/CD pipelines and automated testing
• Use GitFlow or similar version control workflows for development and deployment
• Design and implement innovative analytics solutions using Hadoop, NoSQL, and other Big Data technologies in cloud, on-premise, or hybrid environments
• Build and maintain collaborative partnerships with architects, technical leads, and other key stakeholders
• Conduct detailed analysis of business challenges and technical environments to create effective technical solutions
• Participate in code reviews, test solutions, and write comprehensive project documentation

Requirements

• Minimum of 6+ years of hands-on experience with Big Data technologies and enterprise software development
• Extensive experience with Python, Spark, SQL, and enterprise-level data solutions
• Proven ability to lead technical teams and deliver complex projects
• Deep understanding of cloud-based ecosystems and distributed data systems, including AWS, GCP, and Snowflake
• Proficiency with ETL and streaming pipelines like Apache Airflow and Kafka
• Strong interpersonal and stakeholder management skills to address challenges in transitioning client teams
• Proficiency in CI/CD and DevOps tools such as Git, Docker, and Kubernetes (optional)
• Familiarity with testing practices, including unit testing and component/integration testing
• Understanding of containerized solutions and serverless architecture
• Knowledge of version control workflows (e.g., GitFlow) and Agile methodologies
• Exposure to advanced technologies such as data lake design and distributed systems

We offer

• By choosing EPAM, you're getting a job at one of the most loved workplaces according to Newsweek 2021 & 2022&2023
• Employee ideas are the main driver of our business. We have a very supportive environment where your voice matters
• You will be challenged while working side-by-side with the best talent globally. We work with top-notch technologies, constantly seeking new industry trends and best practices
• We offer a transparent career path and an individual roadmap to engineer your future & accelerate your journey
• At EPAM, you can find vast opportunities for self-development: online courses and libraries, mentoring programs, partial grants of certification, and experience exchange with colleagues around the world. You will learn, contribute, and grow with us

Life at EPAM

• EPAM is a leader in the fastest-growing segment (product development/digital platform engineering) of the IT industry. We acquired Just-BI in 2021 to reinforce our leading position as a global Business Intelligence services provider and have been growing rapidly. With a talented multinational team, we provide data and analytics expertise
• We are currently involved in end-to-end BI design and implementation projects in major national and international companies. We are proud of our entrepreneurial start-up culture and are focused on investing in people by creating continuous learning and development opportunities for our employees who deliver engineering excellence for our clients