Using Elastic Beanstalk with a Docker Registry

Using Elastic Beanstalk with DockerHub or a Docker Registry

Combining DockerHub (using either public or private repositories) or your own Docker registry and Amazon’s Elastic Beanstalk creates a very simple method of delivering a versioned application into a production environment.

The Benefit of Elastic Beanstalk with DockerHub:

For smaller engineering organizations, for organizations running lean on Operations or just staying within the walled-garden of AWS the Elastic Beanstalk <-> DockerHub/Registry combination combines low operational overhead, versioned application rollout/rollback and reliable PaaS service. Assuming you are already building Docker images, the entire setup can be completed within a half of a day.

Overview of Elastic Beanstalk and Docker Deployment Flow:

  1. Build source code and dependencies into a Docker image.
  2. Push an image to DockerHub (use “docker push”)
  3. Push an “Application Version” (use a Dockerfile).
  4. Deploy the application version to a “QA”/”Staging”/”Pre-production” environment, if required.
  5. Deploy the application version to Production

The diagram below is a visual representation of the deployment flow, from start (creating an image) to finish (deployment into Production).

ElasticBeanstalk Deployment from DockerHub

Initial Setup:

If you wish to utilize a private DockerHub repository or your own registry/repository combination and you requires authentication, you’ll need to do a bit of preparation. This preparation is described below.

1. You will need to create and upload a “.dockercfg” file to S3. This file provides authentication information to Amazon’s Elastic Beanstalk. The dockercfg file contains the following text:

{"https://index.docker.io/v1/":{"auth":"3srCWya6OO3y9KUkKLrNZNunIMGiB2Fc","email":"colin@cloudavail.com"}}

You can create a .dockercfg file by running docker login and entering your username and password and the login prompt. This will create a .dockercfg file in the format required by Elastic Beanstalk.

2. If you do use a dockercfg file, the Elastic Beanstalk hosts will need access to it through an AWS IAM profile.

Performing a Deployment:

Upload an Image to DockerHub

Upload Docker image(s) to DockerHub. This makes images available for Elastic Beanstalk or any other host capable of running Docker. A typical Docker push can be done through the command line:

docker push my_organization/my_repository:version (example: cloudavail/test_tomcat_app:0.1.0)

or as a target when using a build tool. For example, I’ve pushed Docker code using Transmode’s gradle-docker plugin.

Create a Dockerrun file for the given “Application Version”

The Dockerrun file is used to describe an application to the Elastic Beanstalk platform. Practically speaking a Dockerun file creates an “Application Version” and instructs Elastic Beanstalk how to do both of the following:

  • “get” an image from DockerHub
  • how to configure a Docker container.

A dockerrun file looks something akin to the following:

{
  "AWSEBDockerrunVersion": "1",
  "Authentication": {
    "Bucket": "my_bucket",
    "Key": "my_dockercfg_file"
  },
  "Image": {
    "Name": "cloudavail/my_tomcat_app:0.1.0",
    "Update": "true"
  },
 "Ports": [
    {
      "ContainerPort": "8080"
    }
  ]
}

The “Authentication” section is required if authentication is needed to access images that are stored in a private repository: “my_bucket” is the name of the bucket where the authentication file is stored and “Key” is the path to the given authentication file. In my example, I’d use something akin to the following

"Bucket": "cloudavail_releases",
 "Key": "docker/dockercfg"

The “Image” section contains the path to a given image and tag.
The “Ports” section contains the port that should be open – which, by default, is mapped to port 80.

Upload the “Application Version” to Elastic Beanstalk

You’ll need to submit the previously created “Dockerrun” file to Amazon. You can do this in one of two ways:

  1. Upload through the UI.
  2. Upload the Dockerrun file to S3 and then issue an API call using Amazon’s CLI tools or custom tooling of your choice.

One cool thing to note – Amazon retains old application versions to allow versioned rollout and rollback. For instance, you can select an application version “0.1.2”, deploy this version to QA, test and then deploy the exact same version to Production. If you needed to rollback, you can select the application version 0.1.1 and deploy this version to Production. The screenshot below demonstrates this:

ElasticBeanstalk - Application Versions

Deploy the newly created Application Version
  1. Deploy through the UI.
  2. Deploy through an API call using Amazon’s CLI tools or custom tooling of your choice.

Criteria for Evaluating Deployment Systems

Criteria for Evaluating Deployment Systems

My goal with this blog post is to provide criteria for evaluating deployment systems – and through this evaluation to either ensure that either the first build of a deployment system or an upgrade of a deployment system meets a company’s needs. The evaluation criteria are broken down into two sections:

  1. User-Facing: attributes of a deployment system that that an application’s user might experience (downtime)
  2. Engineering facing: attributes of a deployment system that Engineering organization will experience (efficiency, ease of deployments).

The deployment system attributes are listed from “Less Mature” (left-most column) to “Most Mature” (right-hand column).

User-Facing (External) Attributes:

The diagram below compares deployment systems from a user’s perspective. Availability is the key measurement. The user only experiences a deployment when an outage or error occurs.

Deployment Systems - User Attributes

  • Availability: can the application function normally during a deployment?
  • Recoverability: if a deployment fails, can the application continue to function normally?

Engineering-Facing (Internal) Attributes:

The diagram below compares deployment systems from an Engineering employee’s perspective. Availability, efficiency, security, scale and transparency are the key measurements.

Deployment Systems - Engineering Attributes

  • Auditing: can the question “who deployed what, where and when” be answered?
  • Dependency Support: does the deployment system also manage resources that the application depends on? Example: does the deployment system manage database schema? Does the deployment system update content delivery network files?
  • Environment Support: how much effort is required to add a new environment? Example: can a “QA2” or “Load Testing” environment be added easily?
  • Re-use of Tooling: can the deployment system be easily re-used to support other applications?
  • Reproducibility: can a given deployment be reproduced? Example: can you give me last week’s release on last week’s Linux server?
  • Rollback: in the event of a failure can the application be rolled back? Can any previous version of the application be deployed?
  • Scale: can the deployment system deploy to any number of nodes?
  • Target Users and Ease of Use: is the deployment system used by both Engineering and Operations or by Operations only?
  • Transparency: does the deployment system integrate with existing tools? Example: are deployment events integrated with an organization’s dashboards?