We are seeking a highly skilled and experienced Senior Big Data Engineer to join an exciting project focused on rearchitecting an on-premises Ruby application, ETL pipelines, and underlying data storage and reports to a cloud-native AWS environment.

Our client is a world-renowned technology company from Silicon Valley, focused on revolutionizing data systems. You will work with world-class experts and technologies in the heart of Silicon Valley, developing products and services that billions of people use every day.

Responsibilities

• Design and implement scalable ETL pipelines in a cloud-native AWS environment
• Develop and maintain data processing workflows using Python, Spark, and Airflow
• Optimize data storage solutions and implement advanced data management strategies using AWS and Redshift
• Ensure the platform meets non-functional requirements by defining robust testing strategies and execution plans
• Collaborate with cross-functional teams to create and maintain decision and change logs, ensuring consistent communication and tracking of technical decisions
• Proactively identify opportunities to enhance the quality of the product while managing scope effectively
• Monitor and fine-tune data pipelines for performance, scalability, and maintainability
• Provide technical guidance and mentorship to team members, promoting best practices in data engineering
• Troubleshoot and resolve complex issues in data workflows and pipelines

Requirements

• 3+ years of experience working in Big Data engineering roles
• Strong data engineering and overall engineering skills, including expertise in testing and quality assurance
• Proficiency in Python
• Background in Spark, Airflow, and AWS technologies like Redshift
• Proven experience in decision-making processes with the ability to document and maintain decision and change logs
• Understanding of strategies to meet non-functional requirements such as performance, scalability, and testing
• Capability to design solutions that enhance product quality without significantly increasing scope
• Excellent command of English (B2+ level), both written and spoken

Nice to have

• Knowledge of Ruby, Iceberg, Trino, Cassandra, and Tableau

We offer

• Career plan and real growth opportunities
• Unlimited access to LinkedIn learning solutions
• International Mobility Plan within 25 countries
• Constant training, mentoring, online corporate courses, eLearning and more
• English classes with a certified teacher
• Support for employee’s initiatives (Algorithms club, toastmasters, agile club and more)
• Enjoyable working environment (Gaming room, napping area, amenities, events, sport teams and more)
• Flexible work schedule and dress code
• Collaborate in a multicultural environment and share best practices from around the globe
• Hired directly by EPAM & 100% under payroll
• Law benefits (IMSS, INFONAVIT, 25% vacation bonus)
• Major medical expenses insurance: Life, Major medical expenses with dental & visual coverage (for the employee and direct family members)
• 13 % employee savings fund, capped to the law limit
• Grocery coupons
• 30 days December bonus
• Employee Stock Purchase Plan
• 12 vacations days plus 4 floating days
• Official Mexican holidays, plus 5 extra holidays (Maundry Thursday and Friday, November 2nd, December 24th & 31st)
• Monthly non-taxable amount for the electricity and internet bills

EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.

By applying to our role, you are agreeing that your personal data may be used as in set out in EPAM´s Privacy Notice and Policy.