Senior Data Engineer (AWS, Databricks)
EPAM Systems
This job is no longer accepting applications
See open jobs at EPAM Systems.See open jobs similar to "Senior Data Engineer (AWS, Databricks)" FinTech Australia.Senior Data Engineer (AWS, Databricks) Description
EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.
We are seeking a remote Senior Data Engineer to join our team and focus on maintaining and implementing new features to the data transformation architecture that is the backbone of our analytical data platform.
In this role, you will be responsible for delivering the high-performance data processing solutions that are efficient and reliable at scale.
Responsibilities
- Provide migration to Unity Catalog
- Customize Databricks Labs UCX tooling to support deployment of existing metastores to Databricks and add features as per requirements
- Drive adoption of newest Databricks metastore to increase security, performance, and enable new features
- Contribute to the metastore design to build compliant, secure, and performant by design architecture
- Create documentation to improve user journey
Requirements
- Bachelor's degree in Computer Science, Software Engineering, or a related field
- Minimum of 3 years of experience in data software engineering
- Proficient knowledge in working with distributed data processing frameworks like Apache Spark
- Hands-on experience with Databricks, Python, PySpark, and SQL
- Expertise in software development processes focusing on testing, continuous integration/continuous delivery (CI/CD), monitoring, and writing documentation
- Understanding of data mesh architecture and data access control mechanisms (e.g. ABAC, RBAC)
- Hands-on experience in cloud technologies (preferably AWS services like IAM, S3, Lambda, EC2)
- Good verbal and written communication skills in English at a B2+ level
Nice to have
- Experience with Hive metastore migration
- Databricks Unity Catalog and Databricks Labs UCX tooling experience
We offer
- We gather like-minded people:
- Engineering community of industry professionals
- Friendly team and enjoyable working environment
- Flexible schedule and opportunity to work remotely within Poland
- Chance to work abroad for up to 60 days annually
- Relocation within our 50+ offices
- We provide growth opportunities:
- Outstanding career roadmap
- Leadership development, career advising, soft skills, and well-being programs
- Certification (GCP, Azure, AWS)
- Unlimited access to LinkedIn Learning, Get Abstract, O’Reilly, Cloud Guru
- Language classes in English and Polish for foreigners
- We cover it all:
- Stable income (Employment Contract or B2B)
- Participation in the Employee Stock Purchase Plan
- Benefits package (health insurance, multisport, shopping vouchers)
- Strategically located offices featuring entertainment and relaxation zones, table tennis and football, free snacks, fantastic coffee, and more
- Referral bonuses
- Corporate, social and well-being events
- Please, note:
- The set of bonuses might vary based on the role you apply for – specifics will be discussed with our recruiter during the general interview
- We will reach out to selected candidates exclusively
This job is no longer accepting applications
See open jobs at EPAM Systems.See open jobs similar to "Senior Data Engineer (AWS, Databricks)" FinTech Australia.