Building ElasticBeanstalk using CloudFormation

The purpose of this blog post is provide the reader an understanding of how to build an ElasticBeanstalk application utilizing a CloudFormation Stack. In particular, the post describes:

  • the resources required in a CloudFormation stack, including the
    • AWS::ElasticBeanstalk::Application Resource
    • AWS::ElasticBeanstalk::Environment Resource
  • the relationship between these resources

After reading this blog post you should be able to build an ElasticBeanstalk application with multiple environments using a CloudFormation file. Subsequent posts will describe methods of deploying code to the ElasticBeanstalk application.

Resources required to support an ElasticBeanstalk Application

Prerequisite Resources

An ElasticBeanstalk application will require some underlying infrastructure such as a VPC, Subnet and, presumably, Internet Gateway, NAT Gateway, Route Tables and Route Table Associations.

Required Resources

AWS::ElasticBeanstalk::Application

The actual ElasticBeanstalk application. An Application serves as a container for Environments and Application Versions.

ElasticBeanstalkApplication:
  Type: AWS::ElasticBeanstalk::Application
  Properties:
    ApplicationName: !Ref AWS::StackName

The screenshot below shows the ElasticBeanstalk Application created by a AWS::ElasticBeanstalk::Application resource.

ElasticBeanstalk - MultipleEnvs - Application.png

AWS::ElasticBeanstalk::Environment

An “Environment” is a subset of the ElasticBeanstalk application. The “Environments” are shown in the AWS Console as parts of an application. For each “Environment” AWS will launch a CloudFormation stack containing components (typically an Auto Scaling Group, ) required to run your application.

ElasticBeanstalkEnvironment:
  Type: AWS::ElasticBeanstalk::Environment
  Properties:
    ApplicationName: !Ref ElasticBeanstalkApplication
    TemplateName: !Ref ElasticBeanstalkConfigurationTemplate

For a given application you will likely have a “Prod” environment or a “QA” environment. In the image below, a CloudFormation file containing two “AWS::ElasticBeanstalk::Environment” resources is used to construct “QA” and “Prod” environments – each of these Environments can have unique configurations and utilize different versions of a codebase.

ElasticBeanstalk - MultipleEnvs - Environments Highlight.png

AWS::ElasticBeanstalk::ConfigurationTemplate

A “Configuration Template” is used to specify the resources required to build an Environment as well as the configuration of these resources. These configuration options include things such as:

  • the “Solution Stack” (where Solution Stack determines what type of AMI will be used to run a given application – for instance, an AMI that contains php, ruby, python or docker)
  • if the Application will utilize an ElasticLoad Balancer
  • the Min Size and Max Size of the Auto Scaling Group supporting the Elastic Beanstalk application if the environment utilizes Auto Scaling
  • the VPC in which an ElasticBeanstalk application should reside if the application resides in a VPC

The full list of “OptionSettings” are available are available here: https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/command-options-general.html. An example Configuration Template is below:

ElasticBeanstalkProdConfigurationTemplate:
  Type: AWS::ElasticBeanstalk::ConfigurationTemplate
  Properties:
    ApplicationName: !Ref ElasticBeanstalkApplication
      OptionSettings:
        - Namespace: aws:autoscaling:asg
          OptionName: MinSize
          Value: 2
        - Namespace: aws:autoscaling:asg
          OptionName: MaxSize
          Value: 2
        - Namespace: aws:autoscaling:launchconfiguration
          OptionName: InstanceType
          Value: t2.micro
        - Namespace: aws:elasticbeanstalk:environment
          OptionName: EnvironmentType
          Value: LoadBalanced
        - Namespace: aws:ec2:vpc
          OptionName: VPCId
          Value: !Ref VPC
        - Namespace: aws:ec2:vpc
          OptionName: Subnets
          # Value: !Join turns the individual subnets into a string
          Value: !Join [ ",", [ !Ref PublicSubnet01, !Ref PublicSubnet02 ] ]
        - Namespace: aws:ec2:vpc
          OptionName: AssociatePublicIpAddress
          Value: true
          SolutionStackName: 64bit Amazon Linux 2018.03 v2.8.4 running PHP 5.6

Link to CloudFormation File

The stack that I used to aid understanding the use of CloudFormation to create AWS ElasticBeanstalk applications is available here: https://github.com/cloudavail/snippets/tree/master/aws/elasticbeanstalk/elasticbeanstalk_with_multiple_envs

Conclusion

If you have any questions about this particular blog post please feel free to post a question below or email blog@cloudavail.com.

Allowing Long Idle Timeouts when using AWS ElasticBeanstalk and Docker

A client I work with had a requirement for a 60 second plus HTTP connection timeout when running Docker on ElasticBeanstalk. Specifically, one of the Engineers was noticing that any HTTP requests taking 60 seconds or more to complete were not being returned by the ElasticBeanstalk application.

Identifying the Cause of the 60 Second Dropped Connections:

The 60 second timeout is actually set in two locations, described below:

  1. The Amazon Elastic Load Balancer, which uses a default “Idle Timeout” value of 60 seconds. The “Idle Timeout” of the given Elastic Load Balancer can be changed easily. (http://docs.aws.amazon.com/ElasticLoadBalancing/latest/DeveloperGuide/config-idle-timeout.html).

    ElasticBeanstalk - ELB - 600 Second Timeout
    ElasticBeanstalk ELB configured with 600 second Idle Timeout.
  2. The nginx Application that acts as a proxy server in front of the Docker container also has a default timeout. The nginx default timeout is not exposed for configuration – you’ll need to modify the nginx configuration through the use of an .ebextensions file or another method. This will also be described within this blog post.
ElasticBeanstalk - HTTP Request Flow
ElasticBeanstalk – HTTP Request Flow

Setting the nginx Timeouts:

The method I used for setting the nginx timeouts can be described, at a high level as:

  1. creating an “ebextension” file that modifies the default nginx configuration used by ElasticBeanstalk. ebextension files are used by Amazon to modify the configuration of ElasticBeanstalk instances.
  2. creating a ZIP format “package” containing a Dockerrun.aws.json file as well as the .ebextension file used to modify the ElasticBeanstalk configuration.

The details are below:

  • Create an “ebextension” file within the root of your project – the file should be at the path .ebextensions/nginx-timeout.config.
  • The content of the file is described below:
files:
  "/etc/nginx/sites-available/elasticbeanstalk-nginx-docker-proxy-timeout.conf":     mode: "000644"
    owner: root
    group: root
    content: |
      proxy_connect_timeout       600;
      proxy_send_timeout          600;
      proxy_read_timeout          600;
      send_timeout                600;
commands:
  "00nginx-create-proxy-timeout":
    command: "if [[ ! -h /etc/nginx/sites-enabled/elasticbeanstalk-nginx-docker-proxy-timeout.conf ]] ; then ln -s /etc/nginx/sites-available/elasticbeanstalk-nginx-docker-proxy-timeout.conf /etc/nginx/sites-enabled/elasticbeanstalk-nginx-docker-proxy-timeout.conf ; fi"
  • Create an application package by running the following from the root of your application’s directory:
    • zip -r ../application-name-$version.zip .ebextensions Dockerrun.aws.json
    • the command above will package the “Dockerrun.aws.json” file as well as the contents of the .ebextensions directory
  • Upload the resulting application-name-$version.zip to AWS ElasticBeanstalk to deploy your application with the nginx timeouts.
  • Note that I’ll be continuing to do testing around the ideal values for the proxy_connect_timeout, proxy_send_timeout, proxy_read_timeout and send_timeout values.

Understanding the “EB Config” sub-command

Understanding the “EB Config” sub-command

The goal of the blog post below is aid a user in better understanding the “EB Config” sub-command – to be perfectly blunt, the eb config documentation does not actually provide enough information to allow a user to utilize the command effectively. A majority of my clients that do use the eb command line interface don’t actually use eb config to save configuration or to commit configuration to version control. Below I’ll describe each config option (delete, list, get, put, save) and what that command actually does.

eb config save $environment_name

Saves a running environment’s configuration in two locations:

  1. An S3 Bucket.
  2. As a YAML file on your own computer.

For instance, if you have an application named “www” and an environment named “www-qa01”, you can save the configuration for this environment by running “eb config save www-qa01.” The following will occur:

  1. User will be prompted to enter a name for the saved configuration – in this example, I’ll use www-qa01
  2. The configuration will be saved in S3 as s3://elasticbeanstalk-us-west-2-187376578462/resources/templates/www/www-qa01 (note that the bucket name depends on your region and account ID and the key name depends on the application name and saved configuration name you provided previously)
  3. The configuration will be copied from S3 to saved to local disk as .elasticbeanstalk/saved_configs/www-qa01.cfg.yml (note that the filename depends on the saved configuration name you provided previously)

A before and after screenshot of running the eb config savecommand: Before command – notice that the AWS Console displays no “Saved Configurations”: eb config save - www-qa01 - Before After command – notice that the eb config save command has written a file to disk and that the AWS Console displays a saved configuration named “www-qa01.” eb config save - www-qa01 - After Lastly – a diagram of how the eb config save command works: eb config save - Diagram

eb config delete $environment_name

As an example, if I ran eb config delete www-qa01, then the following would occur:

  1. The www-qa01 saved configuration object will be removed from S3. As an example, if I ran eb config delete www-qa01, then the object s3://elasticbeanstalk-us-west-2-187376578462/resources/templates/www/www-qa01 would be deleted.
  2. As a result of deleting the www-qa01 saved configuration object from S3, the www-qa01 saved configuration will be removed from AWS Console
  3. The YAML configuration file will be removed from .elasticbeanstalk/saved_configs/www-qa01.cfg.yml
eb config get $environment_name

If I ran eb config get www-qa01, then the following would occur:

  1. If exists, a saved configuration file will be copied from S3 (location: s3://elasticbeanstalk-us-west-2-187376578462/resources/templates/www/www-qa01) to local disk at .elasticbeanstalk/saved_configs/www-qa01.cfg.yml
eb config list

If I ran the eb config list, I will execute a “list” against the objects located at s3://elasticbeanstalk-us-west-2-187376578462/resources/templates/www/*.

eb config put $environment_name

Uploads the named saved configuration to an Amazon S3 bucket. As an example, if I ran eb config put www-qa01, then the following would happen:

  1. A file named .elasticbeanstalk/saved_configs/www-qa01.cfg.yml would be uploaded to S3 at the following location: s3://elasticbeanstalk-us-west-2-187376578462/resources/templates/www/.
  2. As a result of a new object being uploaded to S3, the saved configuration will appear in the AWS Console.

You can also upload a specific file (if not in .elasticbeanstalk/saved_configs/www-qa01.cfg.yml) by running the command as follows: eb config put www-qa01 --cfg ~/path/to/saved_config/www-qa02.config.yml where ~/path/to/saved_config/www-qa02.config.yml is a path to a valid Elastic Beanstalk Configuration file.