Last November, I had the pleasure of announcing the preview of Amazon EC2 Container Service (ECS) at re:Invent. At the time, I wrote about how containerization makes it easier for customers to decompose their applications into smaller building blocks resulting in increased agility and speed of feature releases. I also talked about some of the challenges our customers were facing as they tried to scale container-based applications including challenges around cluster management. Today, I want to dive deeper into some key design decisions we made while building Amazon ECS to address the core problems our customers are facing.

Running modern distributed applications on a cluster requires two key components - reliable state management and flexible scheduling. These are challenging problems that engineers building software systems have been trying to solve for a long time. In the past, many cluster management systems assumed that the cluster was going to be dedicated to a single application or would be statically partitioned to accommodate multiple users. In most cases, the applications you ran on these clusters were limited and set by the administrators. Your jobs were often put in job queues to ensure fairness and increased cluster utilization. For modern distributed applications, many of these approaches break down, especially in the highly dynamic environment enabled by Amazon EC2 and Docker containers. Our customers expect to spin up a pool of compute resources for their clusters on demand and dynamically change the resources available as their jobs change over time. They expect these clusters to span multiple availability zones, and increasingly want to distribute multiple applications - encapsulated in Docker containers - without the need to statically partition the cluster. These applications are typically a mix of long running processes and short lived jobs with varying levels of priority. Perhaps most importantly, our customers told us that they wanted to be able to start with a small cluster and grow over time as their needs grew without adding operational complexity.

A modern scheduling system demands better state management than available with traditional cluster management systems. Customers running Docker containers across a cluster of Amazon EC2 instances need to know where those containers are running and whether they are in their desired state. They also need information about the resources in use and the remaining resources available as well as the ability to respond to failures, including the possibility that an entire Availability Zone may become unavailable. This requires customers to store the state of their cluster in a highly available and distributed key-value store. Our customers have told us that scaling and operating these data storage systems is very challenging. Furthermore, they felt that this was undifferentiated heavy lifting and would rather focus their energy on running their applications and growing their businesses. Let’s dive into the innovations of Amazon ECS that addresses these problems and removes much of the complexity and “muck” of running a high performance, highly scalable Docker-aware cluster management system.

State Management with Amazon ECS

At Amazon, we have built a number of core distributed systems primitives to support our needs. Amazon ECS is built on top of one of these primitives - a Paxos-based transaction journal that maintains a history of state transitions. These transitions are offered and accepted using optimistic concurrency control and accepted offers are then replicated allowing for a highly available and highly scalable ACID compliant datastore. We then expose this state management behind a simple set of APIs. You call the Amazon ECS List and Describe APIs to access the state of your cluster. These APIs give the details of all the instances in your cluster and all the tasks running on those instances. The Amazon ECS APIs respond quickly whether you have a cluster with one instance and a few containers, or a dynamic cluster with 100s of instances and 1000s of containers. There is nothing to install and no database to manage.

Scheduling with Amazon ECS

The state management system underlying Amazon ECS enables us to provide our customers with very powerful scheduling capabilities. Amazon ECS operates as a shared state cluster management system allowing schedulers full visibility to the state of the cluster. The schedulers compete for the resources they need and our state management system resolves conflicts and commits serializable transactions to ensure a consistent and highly available view of cluster state. These transactional guarantees are required to ensure that changes in state are not lost, a very important property to ensure your jobs have the resources they require. This allows scheduling decisions to be made in parallel by multiple schedulers, allowing you to move quickly to create the distributed applications that are becoming increasingly common.

Amazon ECS includes schedulers for common workloads like long running services or run once jobs, and customers can write custom schedulers to meet their unique business or application requirements. This means there is no job queue, no waiting for tasks to start while locks are in place, and that your most important applications are getting the resources they need.

I hope this post has given you some insights into how and why we built Amazon ECS. Developing a system with these capabilities is hard and requires a lot of experience in building, scaling, and operating distributed systems. With Amazon ECS, these capabilities are available to you with just a few API calls. Building modern applications has never been easier. For a walkthrough of the new Amazon ECS Service scheduler and other features, please read Jeff Barr’s post on the AWS blog and for a full list of features and capabilities, read Chris Barclay’s post on the AWS Compute blog