Ask Sendai

The premise behind autoscaling in AWS is simple: you can maximize your ability to handle load spikes and minimize costs if you automatically scale your application out based on metrics like CPU or memory utilization. If you need 100 Docker containers to support your load during the day but only 10 when load is lower at night, running 100 containers at all times means that you’re using 900% more capacity than you need every night.

Kubernetes at CERN: Use Cases, Integration and Challenges. Source: speakerdeck.com

In the previous post, we took a look at the building blocks of Service Mesh Istio, got familiar with the system, and went through the questions that new Istio users often ask. In this post, we will look at how to organize the collection of tracing information over the network. The first thing that developers and system administrators think about when they hear the term Service Mesh is tracing.

So you’ve decided to run your Kubernetes workloads in AWS. As we’ve seen before setting up AWS EKS requires a lot of patience and headache. You may be able to get it working. For others, you should check out the eksctl tool from Weaveworks. Now that you’ve got a Kubernetes cluster, you want to start deploying your microservices to it and start exposing and integrating APIs and services to your clients and other parts of your organization.

This article provides an insight into the creation of the Ambassador open source API gateway for Kubernetes, and discusses the technical challenges and lessons learned from building a developer-focused control plane for managing ingress or ‘edge’ traffic within microservice-based applications. Key Takeaways Developed by Datawire, Ambassador is an open source API gateway designed specifically for use with the Kubernetes container orchestration framework. At its core, Ambassador is a control plane tailored for edge/API configuration for managing the Envoy Proxy “data plane”.

In this blog post, we briefly describe the novel Firmament flow network graph based scheduling approach (OSDI paper) in Kubernetes. We specifically describe the Firmament Scheduler and how it integrates with the Kubernetes cluster manager using Poseidon as the integration glue. We have seen extremely impressive scheduling throughput performance benchmarking numbers with this novel scheduling approach. Originally, Firmament Scheduler was conceptualized, designed and implemented by University of Cambridge researchers, Malte Schwarzkopf & Ionel Gog.

If you are familiar with instrumenting applications, you may have heard of OpenMetrics, OpenTracing, and OpenCensus. These projects aim to create standards for application performance monitoring and collecting metric data. Although the projects do overlap in terms of their goals, they each take a different approach to observability and instrumentation. In this post, we’ll provide an introduction to all three projects, along with some key differentiators of each, and how they best support application monitoring.

How we built and iterated on a machine learning Search Ranking platform for a new two-sided marketplace and how we helped itgrow. Airbnb Experiences are handcrafted activities designed and led by expert hosts that offer a unique taste of local scene and culture. Each experience is vetted for quality by a team of editors before it makes its way onto the platform. We launched Airbnb Experiences in November 2016 with 500 Experiences in 12 cities worldwide.

With 4,000 proprietary microservices and a growing number of open source systems that needed to be monitored, by late 2014 Uber was outgrowing its usage of Graphite and Nagios for metrics. They evaluated several technologies, including Atlas and OpenTSDB, but the fact that a growing number of open source systems were adding native support for the Prometheus Metrics Exporter format tipped the scales in that direction. Uber found with its use of Prometheus and M3, Uber’s storage costs for ingesting metrics became 8.

Configuration management is a hard, unsolved problem. We share some unique insights about the strengths and weaknesses of several popular K8s config management tools. Of all the problems we have confronted, the ones over which the most brainpower, ink, and code have been spilled are related to managing configurations. — Brendan Burns, Brian Grant, David Oppenheimer, Eric Brewer, and John Wilkes, Google Inc. Configuration management is a hard, unsolved problem.