We are seeking a Senior Data Engineer to help our client from the ecommerce industry by upgrading their Databricks and introducing the Unity Catalog.

In this position, you will focus on maintaining and implementing new features to the data transformation architecture, which is the backbone of the analytical data platform. By constantly challenging the status quo, you deliver high-performance data processing solutions that are efficient and reliable at scale.

Join us to leverage your expertise — apply!

Responsibilities

• Migrate data to Unity Catalog
• Customize Databricks Labs UCX tooling to support deployment of existing metastores to Databricks and add features as per requirements
• Drive adoption of newest Databricks metastore to increase security, performance and enabling new features
• Contribute to the metastore design to build compliant, secure and performant by design architecture for 300+ teams and 1000+ users
• Have documentation focus to improve user journey

Requirements

• Proficient knowledge in working with distributed data processing frameworks like Apache Spark
• Understanding of data mesh architecture and data access control mechanisms (e.g ABAC, RBAC)
• Hands-on experience with Databricks
• Proficiency in Python and SQL
• Competency in cloud technologies (preferably AWS services like IAM, S3, Lambda, EC2)
• Engineering craftsmanship with an experience in software development processes focusing on testing, continuous integration/continuous delivery (CI/CD), monitoring and writing documentation

Nice to have

• Databricks Unity Catalog experience
• Databricks Labs UCX tooling experience
• Hive metastore migration experience

We offer

• We gather like-minded people:
  • Engineering community of industry professionals
  • Friendly team and enjoyable working environment
  • Flexible schedule and opportunity to work remotely within Poland
  • Chance to work abroad for up to 60 days annually
  • Relocation within our 50+ offices
• We provide growth opportunities:
  • Outstanding career roadmap
  • Leadership development, career advising, soft skills, and well-being programs
  • Certification (GCP, Azure, AWS)
  • Unlimited access to LinkedIn Learning, Get Abstract, O’Reilly, Cloud Guru
  • Language classes in English and Polish for foreigners
• We cover it all:
  • Stable income (Employment Contract or B2B)
  • Participation in the Employee Stock Purchase Plan
  • Benefits package (health insurance, multisport, shopping vouchers)
  • Strategically located offices featuring entertainment and relaxation zones, table tennis and football, free snacks, fantastic coffee, and more
  • Referral bonuses
  • Corporate, social and well-being events
• Please, note:
  • The set of bonuses might vary based on the role you apply for – specifics will be discussed with our recruiter during the general interview
  • We will reach out to selected candidates exclusively

EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.