Enterprise Event Streaming: Scaling Uber Reserve via Real-Time Kafka Infrastructure

Overview

During my tenure as Product Director at OAG—a global leader in aviation intelligence providing flight schedules and real-time status data worldwide—I led the strategic evolution of our enterprise data delivery platforms. One of our marquee collaborations was with Uber, powering their airport pickup solution, Uber Reserve, which lets riders schedule rides up to 90 days in advance.

To support Uber’s rapid global expansion, I steered our product roadmap to transition away from legacy, on-premise batch APIs toward a modern, cloud-native event streaming platform built on Microsoft Azure. By prioritizing native Apache Kafka compatibility and stream checkpointing to match Uber’s engineering standards, we eliminated high-volume API backfills, cut status update latency from minutes to seconds, and expanded Uber into one of OAG’s strategic premiere accounts.

The Challenge

• High Latency & Rider Friction: Uber’s legacy integration relied on on-premise APIs polling updates within 2-minute batch windows. In ground mobility dispatch, two minutes of stale flight status can mean the difference between a driver curbside on arrival or a costly missed pickup.
• Operational Inefficiency & Missing Checkpoints: The legacy platform lacked stream checkpointing. Any network blip or upstream downtime forced Uber’s engineering team to trigger massive, resource-heavy API backfills to reconcile flight state.
• Reliability & Scaling Bottlenecks: On-premise hosting lacked modern failover redundancy, posing severe availability risks as Uber prepared to scale Reserve from a small North American pilot into hundreds of high-volume international hubs.

Strategy & Product Leadership

• Customer-Driven Roadmap Prioritization: Recognized Uber’s architectural need for Apache Kafka as a strategic market unlock. Instead of enforcing a proprietary Azure-only pattern, I prioritized Kafka protocol support natively over Azure Event Hubs—meeting Uber’s internal architectural standards while positioning OAG’s event platform for wider enterprise adoption.
• Stream Integrity & Custom Checkpointing: Defined functional requirements to bridge Kafka consumer offset tracking with OAG’s sequence numbering and Azure Event Hub checkpointing. This allowed Uber’s backend to cleanly resume streams without missing events or dropping flight state.
• Cloud Resiliency Architecture: Directed the infrastructure migration to a fully cloud-hosted Azure environment backed by triple-redundant Availability Zones (primary region backed by two hot standby zones), eliminating single points of failure.

Impact & Outcomes

• Account Expansion: Commercialized the engagement into an enterprise contract upsell, elevating Uber into one of OAG’s premiere global accounts.
• Sub-Second Latency: Slashed data propagation turnaround from a 2-minute batch delay down to mere seconds from airline status broadcast, enabling real-time curbside driver adjustments.
• Operational Resilience: Stream checkpointing completely eliminated recurring API backfills, drastically reducing engineering overhead and infrastructure costs across both organizations.
• Global Scale: Empowered Uber Reserve’s expansion from 24 North American airports to 111+ international hubs (such as London Heathrow and Amsterdam Schiphol), laying the foundation for 300+ planned locations.

All opinions and perspectives expressed here are solely my own and do not represent the views of my current or past employers.