Ask Sendai

In my previous post (Disaster Recovery for Cloud Solutions is Obsolete) I asserted that you should design your cloud architectures for Disaster Tolerance from the start (even if it is counter intuitive to do so by lean principles). I also argued that you should do this because it’s easy if you do it now, and it will help your business even if there is never a disaster. The problem is that while that’s all true, in practice there are enough gotchas that what should be easy can take you down a lot of rabbit holes before you get to where you need to be.

The last time we wrote about how to save AWS costs was at the end of 2015. AWS has changed a lot since then. AWS introduced AMD-powered EC2 instances that are 10% cheaper compared to the Intel-powered Instances. They provide the same resources (CPU, memory, network bandwidth) and run the same AMIs. The following table shows a mapping from Intel to AMD instance families. You can switch to an AMD family by stopping your EC2 instance, changing the instance type, and starting the instance again.

AWS recently released a new service App Mesh during the 2019 summit which has generated a lot of interest from developers world-wide. This service is a great example of how Amazon is highly customer-focused in delivery of products/features to the market. Besides that, there is no additional charge for using the service!:-) With the advent of cloud, the importance of microservices has increased tremendously. In microservices architecture, large monolithic code-bases/architectures are broken down into smaller, more independent modules.

The idea of a “service mesh” has become increasingly popular over the last couple of years and the number of alternatives available has risen. There are multiple service mesh open-source projects: Istio, Linkerd, Envoy and Conduit which can be deployed on any Kubernetes environment. The AWS App Mesh can be used with microservices running on Amazon Elastic Container Service (Amazon ECS), Amazon Elastic Container Service for Kubernetes (Amazon EKS), and Kubernetes running on Amazon EC2.

AWS App Mesh is now generally available and supported for production use. App Mesh is a service mesh that provides application level networking to make it easy for your services to communicate with each other across multiple types of compute infrastructure. App Mesh standardizes how your services communicate, giving you end-to-end visibility and ensuring high-availability for your applications. Modern applications are typically composed of multiple services. Each service may be built using multiple types of compute infrastructure such as Amazon EC2 and AWS Fargate.

Last week we made a fairly quiet (too quiet, in fact) announcement of our plan to slowly and carefully deprecate the path-based access model that is used to specify the address of an object in an S3 bucket. I spent some time talking to the S3 team in order to get a better understanding of the plan. We launched S3 in early 2006. Jeff Bezos’ original spec for S3 was very succinct – he wanted malloc (a key memory allocation function for C programs) for the Internet.

While evaluating the performance of a server application, we eventually (and hopefully) run up against the most fundamental constraining factor: the network. Cloud providers tend to offer somewhat handwavy guidance on networking constraints, especially when compared to the exhaustive literature explaining the quotas for RAM, CPU, and I/O. While working on an unrelated stress test in EC2, we were surprised by some results that led us down the path of investigating EC2 network capacity claims, resulting in this writeup.

The premise behind autoscaling in AWS is simple: you can maximize your ability to handle load spikes and minimize costs if you automatically scale your application out based on metrics like CPU or memory utilization. If you need 100 Docker containers to support your load during the day but only 10 when load is lower at night, running 100 containers at all times means that you’re using 900% more capacity than you need every night.

Our security culture is one of the things that sets AWS apart. Security is job zero — it is the foundation for all AWS employees and impacts the work we do every day, across the company. And that’s reflected in our services, which undergo exacting internal and external security reviews before being released. From there, we have historically waited for customer demand to begin the complex process of third-party assessment and validating services under specific compliance programs.

The Guardian migrated their CMS’s datastore in 2018 from a self-managed MongoDB cluster to PostgreSQL on Amazon RDS for a fully managed solution. The team did an API-based migration without any downtime. Guardian’s in-house CMS – called Composer – which stores articles, blog content, photo galleries and video was originally built on top of MongoDB as a datastore. This was preceded by a vendor software backed by an Oracle database.