Ask Sendai

As a researcher on Computer Vision, I come across new blogs and tutorials on ML (Machine Learning) every day. However, most of them are just focussing on introducing the syntax and the terminology relevant to the field. For example – a 15 minute tutorial on Tensorflow using MNIST dataset, or a 10 minute intro to Deep Learning in Keras on Imagenet. While people are able to copy paste and run the code in these tutorials and feel that working in ML is really not that hard, it doesn’t help them at all in using ML for their own purposes.

Deep learning models are large and complex, so instead of writing out every function from the ground up, programmers rely on frameworks and software libraries to build neural networks efficiently. The top deep learning frameworks provide highly optimized, GPU-enabled code that are specific to deep neural network computations. Source: nvidia.com

Episodic Curiosity through Reachability: Observations are added to memory, reward is computed based on how far the current observation is from the most similar observation in memory. The agent receives more reward for seeing observations which are not yet represented in memory. Source: googleblog.com

Running serverless applications on Kubernetes is quite new. Most of the options available are still under major development. You can find the ever growing number of FaaS platforms that run on top of Kubernetes in our FaaS category. At lot of the value proposition people are seeing from using AWS Lambda, to quickly string together batch operations, don’t transfer well to Kubernetes. It is currently quicker and easier to simply use containers for this scenario.

Open source software pervades the work we do at Uber. On the infrastructure side, we have contributed projects like Jaeger, which lets engineers trace complex architectures, and M3, a metrics platform that works with Prometheus. For front-end development, we built RIBs, a cross-platform architecture for mobile apps, along with Fusion.js, a plugin-based web framework. In the rapidly advancing area of machine learning, we have open source tools such as Horovod, a distributed training framework, and Pyro, a deep probabilistic programming language written in Python.

As a company heavily invested in AI, Uber aims to leverage machine learning (ML) in product development and the day-to-day management of our business. In pursuit of this goal, our data scientists spend considerable amounts of time prototyping and validating powerful new types of ML models to solve Uber’s most challenging problems (e.g., NLP based smart reply systems, ticket assistance systems, fraud detection, and financial and marketplace forecasting). Once a model type is empirically validated to be best for the task, engineers work closely with data science teams to productionize and make it available for low latency serving at Uber-scale.

We’ve been running Kubernetes for deep learning research for over two years. While our largest-scale workloads manage bare cloud VMs directly, Kubernetes provides a fast iteration cycle, reasonable scalability, and a lack of boilerplate which makes it ideal for most of our experiments. We now operate several Kubernetes clusters (some in the cloud and some on physical hardware), the largest of which we’ve pushed to over 2,500 nodes. This cluster runs in Azure on a combination of D15v2 and NC24 VMs.

High quality map data powers many aspects of the Uber trip experience. Services such as Search, Routing, and Estimated Time of Arrival (ETA) prediction rely on accurate map data to provide a safe, convenient, and efficient experience for riders, drivers, eaters, and delivery-partners. However, map data can become stale over time, reducing its quality. As a customer-obsessed company, Uber reviews and addresses feedback in customer support tickets, which are submitted by riders, driver-partners, eaters, and delivery-partners on the Uber platform.

Are self-healing applications on our horizon? CA Technologies’ new AIOps platform hopes to bring us closer to the future of intelligent automation. What are the grand plans for AIOps and is it a potential reality or just another buzzword? Artificial Intelligence for IT Operations, also known as AIOps, gains momentum as our machine learning capabilities and algorithims become more complex. On October 16, 2018 CA Technologies revealed a new AIOps platform that will help teams automate tasks intelligently.

This time around, I am happy to revisit two of the most important features of Kubernetes 1.12 and take a closer look at what’s under their hood and what the future prospectsare. Source: jaxenter.com