Tjippies · Airbnb
Staff Software Engineer, Streaming Infrastructure
Description
Airbnb was born in 2007 when two hosts welcomed three guests to their San Francisco home, and has since grown to over 5 million hosts who have welcomed over 2 billion guest arrivals in almost every country across the globe. Every day, hosts offer unique stays and experiences that make it possible for guests to connect with communities in a more authentic way. Airbnb’s Data Infrastructure organization builds the platforms that power data across the company. Within it, the Data Movement & Transformation (DMT) organization owns the platform that moves data between Airbnb’s online databases, event streams, and data warehouse, and transforms it along the way. Fraud prevention, real-time reporting, search, and AI products all depend on it. Much of that data still moves through bespoke pipelines that each team builds and runs differently. We’re replacing them with one declarative, self-serve platform. You define your data once, and the platform provisions, runs, and evolves the pipelines that deliver it wherever it needs to go. Hundreds of production pipelines already run on it. Next, we’re bringing every data movement use case at Airbnb onto one platform, with a simpler architecture underneath and a single entry point for every engineer. DMT is hiring Staff Software Engineers across four teams: Data Frameworks: the declarative pipeline platform and its control plane, including runtime selection across streaming and batch, a growing library of source and sink connectors, and end-to-end lineage. Ingestion: change data capture (CDC) and snapshot exports from Airbnb’s online databases into the warehouse, focused on data completeness and self-serve onboarding. Realtime Data Infrastructure: the Kafka and Flink platforms that real-time data flows through, running on Kubernetes at Airbnb scale. Schema Management: how schemas are defined, registered, and safely evolved across streaming, storage, and warehouse systems. As a Staff Software Engineer, you’ll set technical direction for your area, stay hands-on in the code, and lead work that spans teams. Examples of the Difference You’ll Make Unify data movement at Airbnb. Shape the declarative platform that replaces bespoke pipelines across the company. Engineers define their data once, and the platform picks the right streaming or batch runtime and delivers it to every system that needs it. Build the data foundation for AI. Airbnb’s AI and agent platforms are building on DMT instead of rolling their own pipelines. You’ll make sure fresh, trustworthy, well-documented data reaches them at the speed those products need. Run real-time infrastructure that survives anything. Design Kafka and Flink platforms that keep serving through a full regional outage, fail over within minutes, and have no single points of failure at their core. Make schemas a contract across every system. Build schema management where a change is validated and propagated end to end across streaming, storage, and warehouse systems, with no human in the loop. Guarantee data correctness at scale. Make sure data arrives complete and accurate, with zero data loss, column-level lineage, and access controls the business relies on for fraud prevention, reporting, and compliance. Make data infrastructure self-serve. Give every engineer at Airbnb a single entry point to move data, with onboarding measured in hours instead of days and AI-assisted tools for debugging and operations. Raise the technical bar. Set direction through design reviews, lead incident follow-through, and grow the engineers around you. Your Expertise 9+ years of software engineering, including significant experience building and operating large-scale distributed systems, ideally data platforms or streaming infrastructure. Deep expertise in at least one of the following areas, and comfort learning the others: Kafka or a similar distributed log: partitioning, delivery semantics, replication, and multi-region operation Stream or batch processing such as