Senior Data Software Engineer
EPAM Systems
Software Engineering
Portugal
Posted on Dec 23, 2024
Senior Data Software Engineer Description
We are seeking a Senior Data Software Engineer to join our dynamic team.
In this role, you will design and develop robust data solutions to cater to our extensive consumer base. You will leverage your expertise in Python, PySpark, and Azure technologies to propel our products towards greater efficiency and scalability.
#LI-DNI
Responsibilities
- Design and develop scalable data products for a large consumer base
- Build and optimize data pipelines using Python and PySpark
- Implement and maintain Azure Databricks environments
- Utilize Azure DevOps pipelines and/or GitHub Actions for effective deployment of data solutions
- Collaborate with cross-functional teams to translate business needs into technical specifications
- Ensure adherence to software engineering best practices throughout the project lifecycle
- Contribute to the development and maintenance of the Databricks Unity Catalog
- Optimize data processing and storage solutions within the Azure data landscape
Requirements
- At least 3 years of relevant working experience in data software engineering
- Proficiency in Python and PySpark with a solid track record of developing large-scale data products
- Competency in deploying and managing solutions using Azure Databricks
- Background in managing pipelines with Azure DevOps and/or GitHub Actions
- Understanding of the Azure data landscape, including Azure Event Hubs and Terraform
- Fluent English communication skills at a B2+ level
Nice to have
- Familiarity with developing APIs using Python
- Working experience with Google Cloud Platform (GCP)
We offer
- Competitive compensation depending on experience and skills
- Variety of projects within one company
- Being a part of a project following engineering excellence standards
- Individual career path and professional growth opportunities
- Internal events and communities
- Flexible work hours