Decoding NLP Attention Mechanisms

In this blogpost series, we will walk you through the rise of the transformer architecture. Our first stop will be a focus on the attention mechanism, which is the key component of this architecture. We will then move on to the transformer itself in part II, and finally, we’ll introduce BERT in part III. In order to understand the transformer and its motives, we will need to dive into its core idea: a novel paradigm called attention.
Read more

Is your machine learning/quantum computer lying to you?

If you ask a machine learning model to give you an answer, how would you know if it’s skewed? How would you know if the inputs or outputs have been tampered with? RSA 2020 – Is your machine learning/quantum computer lying to you? And how would you know if the algorithm was tampered with? Quantum computing is the new heir apparent to AI in terms of number of wild security claims.
Read more

Container-Native Multi-Cluster Global Load Balancing on GCP

During a recent project, a Jetstack customer wanted to load balance global traffic to multiple Google Kubernetes Engine (GKE) clusters, while also benefiting from Google’s Cloud Armor to protect against denial of service (DoS) attacks. Additionally, they wanted to make use of container-native load balancing for improved traffic visibility and network performance. Google Cloud Platform (GCP) offers various load balancing solutions which are generally well documented and easy to use.
Read more

The Winding Road to Better Machine Learning Infrastructure Through Tensorflow Extended andKubeflow

When Spotify launched in 2008 in Sweden, and in 2011 in the United States, people were amazed that they could access almost the world’s entire music catalog instantaneously. The experience felt like magic and as a result, music aficionados dug in and organized that content into millions of unique playlists. Early on, our users relied on playlists and rudimentary recommendation features like a related artists feature to surface new music.
Read more

Project LightSpeed: Rewriting the Messenger codebase

To build this new version of Messenger, we needed to rebuild the architecture from the ground up and rewrite the entire codebase. We are excited to begin rolling out the new version of Messenger on iOS. To make the Messenger iOS app faster, smaller, and simpler, we rebuilt the architecture and rewrote the entire codebase, which is an incredibly rare undertaking and involved engineers from across the company. Compared with the previous iOS version, this new Messenger is twice as fast to start and is one-fourth the size.
Read more

Machine learning beats BTC/USDT on unseen data, even with transaction fees and slippage.

Machine learning beats BTC/USDT on unseen data, even with transaction fees and slippage. There are a lot of articles about experimentation applying machine learning to crypto trading but it is hard to find one with realistic methodology. Ideally, results should come either from trading history on an actual exchange or from a simulation with unseen data and included transaction fees. That’s why I wrote this article — I want to tell you how I do my financial market research, present some of its findings and eventually show you actual results.
Read more

Gmail Is Catching More Malicious Attachments With Deep Learning

Users of Gmail get 300 billion attachments each week. To separate legitimate documents from harmful ones, Google turned to AI—and it’s working. Distributing malware by attaching tainted documents to emails is one of the oldest tricks in the book. It’s not just a theoretical risk—real attackers use malicious documents to infect targets all the time. So on top of its anti-spam and anti-phishing efforts, Gmail expanded its malware detection capabilities at the end of last year to include more tailored document monitoring.
Read more

Reflections on software performance

At this point in my career, I’ve worked on at least three projects where performance was a defining characteristic: Livegrep, Taktician, and Sorbet (I discussed sorbet in particular last time, and livegrep in an earlier post). I’ve also done a lot of other performance work on the tools I use, some of which ended up on my other blog, Accidentally Quadratic. In this post, I want to reflect on some of the lessons I’ve learned while writing performant software, and working with rather a lot more not-so-performant software.
Read more

How Allstate’s Secret Auto Insurance Algorithm Squeezes Big Spenders

Seven years ago, Allstate Corporation told Maryland regulators it was time to update its auto insurance rates. The insurer said its new, sophisticated risk analysis showed it was charging nearly all of its 93,000 Maryland customers outdated premiums. Some of the old rates were off by miles. One 36-year-old man from Prince George’s County, Md., who Allstate said in public records should have been paying $3,750 every six months, was instead being charged twice that, more than $7,500.
Read more

Driver stranded after connected rental car can’t call home

Over the weekend, a trip to the Californian boonies by Guardian journalist Kari Paul turned into a cautionary tale about the perils of the connected car and the Internet of Things. Paul had rented a car through a local car-sharing service called GIG Car Share, which offers a fleet of hybrid Toyota Priuses and electric Chevrolet Bolt EVs in the Bay Area and Sacramento, with plans to spend the weekend in a more rural part of the state about three hours north of Oakland.
Read more