FinTech Australia
FinTech Australia
About
About Us
What is Fintech
Contact Us
Policy
Policy
Policy Working Groups
Events
Events Calendar
The Finnies
Intersekt Festival
Members
Corporate Partners
Fintech Careers
Jobs Board
eLearning
Resources
Ecosystem Map
Regulatory Map
Investor Map
EY Fintech Census
Services Directory
News
News
Podcast
Member Portal
FinTech Australia
FinTech Australia
About
About Us
What is Fintech
Contact Us
Policy
Policy
Policy Working Groups
Events
Events Calendar
The Finnies
Intersekt Festival
Members
Corporate Partners
Fintech Careers
Jobs Board
eLearning
Resources
Ecosystem Map
Regulatory Map
Investor Map
EY Fintech Census
Services Directory
News
News
Podcast
Member Portal
Folder: About
Folder: Policy
Folder: Events
Members
Corporate Partners
Folder: Fintech Careers
Folder: Resources
Folder: News
Member Portal
Back
About Us
What is Fintech
Contact Us
Back
Policy
Policy Working Groups
Back
Events Calendar
The Finnies
Intersekt Festival
Back
Jobs Board
eLearning
Back
Ecosystem Map
Regulatory Map
Investor Map
EY Fintech Census
Services Directory
Back
News
Podcast
hero

Companies you'll love to work for

131
companies
7,168
Jobs
For Employers
Add your job
listings
Contact Us
For Employers
Find Candidates
Directly
Talent Pool
For Candidates
Help Recruiters
Find You
Talent Network
Search 
jobs
Explore 
companies
Join talent network
Talent
My job alerts

Staff Software Engineer - Distributed Data Systems

Adyen

Adyen

Software Engineering
Amsterdam, Netherlands
Posted on Apr 10, 2025
Apply now

This is Adyen

Adyen provides payments, data, and financial products in a single solution for customers like Meta, Uber, H&M, and Microsoft - making us the financial technology platform of choice. At Adyen, everything we do is engineered for ambition.

For our teams, we create an environment with opportunities for our people to succeed, backed by the culture and support to ensure they are enabled to truly own their careers. We are motivated individuals who tackle unique technical challenges at scale and solve them as a team. Together, we deliver innovative and ethical solutions that help businesses achieve their ambitions faster.

Staff Software Engineer - Distributed Data Systems

Adyen hosts a significant footprint in both scale and variety of distributed systems. This covers distributed compute (Spark, Trino, Flink), distributed databases (Cassandra, Druid), and distributed file/object storage (HDFS, Ceph Object Gateway). These technologies are offered as-a-service internally, towards product engineering, to support them in building and scaling world-class products.

We’re looking for an expert with deep knowledge in distributed systems, to both improve operations/scalability of existing offerings, as well as introduce and mature new ones. The initial focus will be scaling and tuning our Hadoop and Spark infrastructure, in addition to iterating on our OLAP platform, where we currently use Apache Druid. Over time, we expect this expertise to be useful in a wider range of distributed systems internally.

What you’ll do

You’ll be asked to both build new as-a-service offerings, and improve the existing ones. This role is perfect for you if you're passionate about one or all of the following:

  • Design, build, and optimize data clusters to ensure scalability, fault tolerance, and high availability. Covering both batch and streaming workloads.
  • Focus on improving the Spark, Hadoop, Kubernetes, Deltalake, and Druid ecosystems internally.
  • Evolve these open-source projects internally, with the intention of contributing this code upstream.
  • Educate and grow Adyen’s internal knowledge in these topics. Covering both peer platform engineers, and the platform users.

Who you are

Must have experience in:

  • Scaling and tuning large deployments of Spark-on-k8s and Spark-on-Hadoop
  • Hadoop and the HDFS protocol
  • Designing and tuning shuffle heavy systems, on yarn, or on k8s via remote shuffle services
  • One of the lakehouse file formats (Delta, Iceberg, Hudi)
  • OLAP technologies covering at least one of Clickhouse, Apache Druid, Apache Pinot, or Apache Doris.
  • Open-source contributions in one of the must-have technologies, or other common ones (e.g. Kafka, Cassandra, Trino, etc)
  • Team player with strong communication skills
  • Ability to work closely with diverse stakeholders you enable (analysts, data scientists, data engineers, etc.) and depend upon (infrastructure, security, etc).
  • Demonstrated ability to troubleshoot and resolve issues in large-scale, production environments with distributed systems.

    Nice-to-have experience in:

  • Next generation and multi-modal data formats (e.g. LanceDB)
  • Building self-service stateful platforms
  • Open-source S3 alternatives (e.g. ceph, minio, etc)
  • Native or accelerated runtimes for Spark (Apache DataFusion Comet, Apache Gluten, Nvidia RAPIDS, etc)

Our Diversity, Equity and Inclusion commitments

Our unique approach is a product of our diverse perspectives. This diversity of backgrounds and cultures is essential in helping us maintain our momentum. Our business and technical challenges are unique, and we need as many different voices as possible to join us in solving them - voices like yours. No matter who you are or where you’re from, we welcome you to be your true self at Adyen.

Studies show that women and members of underrepresented communities apply for jobs only if they meet 100% of the qualifications. Does this sound like you? If so, Adyen encourages you to reconsider and apply. We look forward to your application!

What’s next?

Ensuring a smooth and enjoyable candidate experience is critical for us. We aim to get back to you regarding your application within 5 business days. Our interview process tends to take about 4 weeks to complete, but may fluctuate depending on the role. Learn more about our hiring process here. Don’t be afraid to let us know if you need more flexibility.

This role is based out of our Amsterdam office. We are an office-first company and value in-person collaboration; we do not offer remote-only roles.

Apply now
See more open positions at Adyen
Privacy policyCookie policy
FINTECH AUSTRALIA

FinTech Australia exists to help our country become one of the world’s top markets for fintech innovation and investment.

IMPORTANT LINKS
  • Privacy Policy
  • Member Login
  • Join Fintech Australia
  • Contact Us
© 2023 FinTech Australia