Ask Sendai

At LinkedIn, we like to say that experimentation is in our blood because no production release at the company happens without experimentation; by “experimentation,” we typically mean “A/B testing.” The company relies on employees to make decisions by analyzing data. Experimentation is a data-driven foundation of the decision-making process, which helps with measuring the precise impact of every change and release, and evaluating whether expectations meet reality. LinkedIn’s experimentation platform operates at an extremely large scale: It serves up to 800,000 QPS of network calls, It serves about 35,000 concurrently running A/B experiments, It handles up to 23 trillion experiment evaluations per day, Average latency of experiment evaluation is 700 ns and the 99th percentile is 3 μs, It is used in about 500 production services.

Read the 2020 Cloud Report from Cockroach Labs, and learn which cloud platform performs best for transactional workloads across TPC-C, Network Throughput, CPU, and Storage benchmarks. If there’s one thing we’ve learned in our three years of benchmarking cloud providers on transactional workloads, it’s this: the results change often. Last year’s report showed AWS dramatically outperforming GCP across TPC-C performance, CPU, Network, and even cost. The 2020 Cloud Report shows GCP caught up to AWS’s performance and offers the best price per performance for transactional workloads, and that new-to-the-report Azure is broadly competitive with both GCP and AWS.

Andrey Zolotov, Gideon Low present their journey of transition to distributed data processing using GemFire and the challenges faced along the way. Source: infoq.com

Kubernetes, Istio, knative and an internally developed specification for “hardening” containers are now the default software development platform across the military. Just like almost everything else, military organizations increasingly depend on software, and they are turning to an array of open source cloud tools like Kubernetes and Istio to get the job done, according to a presentation delivered by Nicholas Chaillan, chief software officer for the U.S. Air Force, at KubeCon 2019 in San Diego.

The traditional answer is that hash tables are designed to be efficient when storing data in memory, while B-Trees are designed for slower storage that is accessed in blocks. However, this is not a fundamental property of these data structures. There are hash tables designed to be used on disk (e.g. MySQL’s hash index), many in-memory trees (e.g. Java’s TreeMap, C++’s map), and even in-memory B-Trees. I think the most important answer is that B-Trees are more ‘general purpose,’ which results in lower ‘total cost’ for very large persistent data.

Undersea cables are responsible for moving data between countries and continents at high speeds, making everything from photo sharing to financial transactions possible. These cables use fiber optics to move data at high speeds to land, where the data is then conveyed via fiber optics to homes and businesses. Yet, despite the billions of people relying on the data moved by undersea cables, there are only about 380 of them worldwide as of 2019, according to CNN estimates, though they span more than 745,000 miles—or more than three times the distance to the moon.

A new method determines whether circuits are accurately executing complex operations that classical computers can’t tackle. Researchers from MIT, Google, and elsewhere have designed a protocol called Variational Quantum Unsampling, based on a novel quantum neural network, that verifies when photonic Noisy Intermediate Scale Quantum (NISQ) chips have accurately performed complex computations using light. Source: mit.edu

In 5 years, the number of endpoints consumed by Lyft’s mobile apps grew to over 500, and the size of our mobile engineering team increased by more than 15x. To scale with this growth, our infrastructure had to evolve dramatically to utilize new advances in modern networking in order to continue to provide benefits for our users. This post describes the journey through the evolution of Lyft’s mobile networking: how it’s changed, what we’ve learned, and why it’s important for us as a growing business.

Using the same kind of technology that translates English into Mandarin, the neural network translates problems into solutions Despite being built by calculus, linear algebra, and an army of statisticians around the world, neural networks have trouble understanding math. Or at least, they have trouble understanding how humanity writes math equations. Facebook’s AI research team, however, claims to have developed a new approach to turn complex math problems into machine-readable data.

Lidar sensors work by bouncing laser light off surrounding objects to produce a three-dimensional ‘point cloud.’ The first modern three-dimensional lidar was created for the 2005 DARPA Grand Challenge, a pivotal self-driving car competition. Today, many experts continue to see lidar as a key enabling technology for self-driving cars. That original 2005 lidar, made by a company called Velodyne, contained a vertical array of 64 lasers that spun around 360 degrees.