At Uber, we use robust data processing systems such as Apache Flink and Apache Spark to power the streaming applications that helps us calculate up-to-date pricing, enhance driver dispatching, and fight fraud on our platform. Such solutions can process data at a massive scale in real time with exactly-once semantics, and the emergence of these […]
Mayank Bansal and Min Cai present Peloton, a Unified Resource Scheduler for collocating heterogeneous workloads in shared Mesos clusters. Its goal is to manage compute resources more efficiently while providing hierarchical max-min fairness guarantees for different teams. Peloton schedules large-scale batch jobs with millions of tasks and supports distributed TensorFlow jobs with thousands of GPUs. […]
Deriving information and insights from data in the Uber marketplace requires analyzing data across an entire city. Grid systems, which partition geographic areas into identifiable cells and facilitate the exploration of data at a fine granularity, are critical to this effort. When Uber designed a grid system to bucket geospatial events, we looked to the […]
Uber AI Labs introduces a method for making neural networks that process images faster and more accurately by leveraging JPEG representations. Neural networks, an important tool for processing data in a variety of industries, grew from an academic research area to a cornerstone of industry over the last few years. Convolutional Neural Networks (CNNs) have […]
Uber, like most large technology companies, relies extensively on metrics to effectively monitor its entire stack. From low-level system metrics, such as memory utilization of a host, to high-level business metrics, including the number of Uber Eats orders in a particular city, they allow our engineers to gain insight into how our services are operating […]
To ensure the stable, scalable growth of our diverse tech stack, we leverage a microservices-oriented architecture, letting engineers deploy thousands of services on a dynamic, high-velocity release cycle. These services enable new features to greatly improve the experiences of riders, drivers, and eaters on our platform. Although this paradigm supported hypergrowth in both scale and […]
During a September 2018 meetup, Uber’s Payments Platform team discusses how this technology supports our company’s growth through an active-active architecture, exactly-once payment processing, and scalability across businesses. From ridesharing, food delivery, and new modalities to freight, healthcare, and business travel, our technology helps create billions of moments of human connection around the world. Building […]
Uber’s many data flows required modeling the data associated with a specific task, such as a rider trip, into a state machine. The state machine lets engineers focus on just the events needed to successfully accomplish a trip. In one sense, Uber’s challenge of efficiently matching riders and drivers in the real world comes down […]