Peloton – Uber’s Webscale Unified Scheduler on Mesos & Kubernetes

Mayank Bansal and Min Cai present Peloton, a Unified Resource Scheduler for collocating heterogeneous workloads in shared Mesos clusters. Its goal is to manage compute resources more efficiently while providing hierarchical max-min fairness guarantees for different teams. Peloton schedules large-scale batch jobs with millions of tasks and supports distributed TensorFlow jobs with thousands of GPUs. […]

Visualizing City Cores with H3, Uber’s Open Source Geospatial Indexing System

Deriving information and insights from data in the Uber marketplace requires analyzing data across an entire city. Grid systems, which partition geographic areas into identifiable cells and facilitate the exploration of data at a fine granularity, are critical to this effort. When Uber designed a grid system to bucket geospatial events, we looked to the […]

Faster Neural Networks Straight from JPEG

Uber AI Labs introduces a method for making neural networks that process images faster and more accurately by leveraging JPEG representations. Neural networks, an important tool for processing data in a variety of industries, grew from an academic research area to a cornerstone of industry over the last few years. Convolutional Neural Networks (CNNs) have […]

Introducing Makisu: Uber’s Fast, Reliable Docker Image Builder for Apache Mesos and Kubernetes

To ensure the stable, scalable growth of our diverse tech stack, we leverage a microservices-oriented architecture, letting engineers deploy thousands of services on a dynamic, high-velocity release cycle. These services enable new features to greatly improve the experiences of riders, drivers, and eaters on our platform. Although this paradigm supported hypergrowth in both scale and […]

Engineering Uber’s Next-Gen Payments Platform

During a September 2018 meetup, Uber’s Payments Platform team discusses how this technology supports our company’s growth through an active-active architecture, exactly-once payment processing, and scalability across businesses. From ridesharing, food delivery, and new modalities to freight, healthcare, and business travel, our technology helps create billions of moments of human connection around the world. Building […]

Sessionizing Uber Trips in Real Time

Uber’s many data flows required modeling the data associated with a specific task, such as a rider trip, into a state machine. The state machine lets engineers focus on just the events needed to successfully accomplish a trip. In one sense, Uber’s challenge of efficiently matching riders and drivers in the real world comes down […]