Reducing AWS Cost using Scheduled Scaling

Reducing AWS Cost using Scheduled Scaling

One of the ways to reduce AWS cost is to utilize Auto Scaling Groups and the “Scheduled Scaling” feature to scaled down EC2 resources during non-business hours. In the example below, I’ll walk you through the use of scheduled scaling to “Scale Down” an api-qa01 Auto Scaling Group to 0 instances after the close of business (assumed to be 7:00 pm pacific) and “Scale Up” an api-qa01 Auto Scaling Group to two instances at start of business (assumed to be 8:00 am pacific).

Setup Scheduled Scaling:

To setup scheduled scaling, you’ll need:

  1. Infrastructure that uses Auto Scaling Groups (you could do something similar with EC2 start/stop instances, but I don’t plan to cover this case in the blog). It would be ideal if you’ve configured your Auto Scaling Group to publish Auto Scaling Group Metrics – this will let you track the capacity of an Auto Scaling Group. To do so:
    1. Check “Enable CloudWatch detailed monitoring” when creating the Launch Configuration in the AWS Console, or use –instance-monitoring when creating a Launch Configuration using the AWS Command Line Interface.
    2. Use “aws autoscaling enable-metrics-collection” or check “Enable CloudWatch detailed monitoring” when creating the Auto Scaling Group from the AWS Console.
  2. A computer with the “AWS Command Line Interface” tools installed (scheduled scaling is not yet available in the AWS Console)

With the above prerequisites set, configuring scheduled scaling is as simple as doing the following:

  1. Identifying the name of the group for which you wish to enable scheduled scaling.
  2. Creating a “Scale Up” scheduled event, example below:
    1. aws autoscaling put-scheduled-update-group-action --scheduled-action-name scaleup-api-qa01 --auto-scaling-group-name api-qa01 --recurrence "0 15 * * *" --desired-capacity 2
  3. Creating a “Scale Down” scheduled event:
    1. aws autoscaling put-scheduled-update-group-action --scheduled-action-name scaledown-api-qa01 --auto-scaling-group-name api-qa01 --recurrence "0 2 * * *" --desired-capacity 0
  4. Once you done creating “Scale Up” and “Scale Down” events, you’ll want to ensure that you’ve setup the scheduled scaling actions correctly – the command to do this is below:
    1. aws autoscaling describe-scheduled-actions --auto-scaling-group-name api-qa01 --query 'ScheduledUpdateGroupActions[*].{Name:ScheduledActionName,DesiredCapacity:DesiredCapacity,Recurrence:Recurrence}' --output table
    2. The result should look something akin to the followingScheduled Scaling - Described Scheduled Actions
Confirming Scaling:

If you’ve enabled Auto Scaling Group Metrics you should be able to identify the changes in Desired Capacity and in InService Capacity. An example of the Auto Scaling Group api-qa01 is shown below:

Scheduled Scaling Example

Noteice that the “GroupDesiredCapacity” and “GroupTotalInstances” increases to 2 Instances daily at 15:00 UTC and then returns to 0 Instances daily at 2:00 UTC.

If You Don’t Use Auto Scaling:

Those not using Scheduled Scaling might be able to make be able to utilize a cron job that Starts and Stops EC2 instances based on a Tag. For instance, you could create Tags with the Key “StartAfter” and “StopAfter” and then utilize the Values of these tags to inform a cron job when to Start/Stop EC2 instances.

In Summary:
  1. You should have everything you need in order to schedule Start/Stop of instances based on time of day.
  2. Use of scheduled Scale Up / Scale Down is a great way to ensure that your staff is familiar with Auto Scaling (QA or Developers might need to Scale Up infrastructure outside of business horus) and that your AWS Infrastructure is capable of withstanding outages or periods of time when no EC2 instances are running.
  3. If you’ve got questions, please feel free to comment below or send me an email.

Understanding the “EB Config” sub-command

Understanding the “EB Config” sub-command

The goal of the blog post below is aid a user in better understanding the “EB Config” sub-command – to be perfectly blunt, the eb config documentation does not actually provide enough information to allow a user to utilize the command effectively. A majority of my clients that do use the eb command line interface don’t actually use eb config to save configuration or to commit configuration to version control. Below I’ll describe each config option (delete, list, get, put, save) and what that command actually does.

eb config save $environment_name

Saves a running environment’s configuration in two locations:

  1. An S3 Bucket.
  2. As a YAML file on your own computer.

For instance, if you have an application named “www” and an environment named “www-qa01”, you can save the configuration for this environment by running “eb config save www-qa01.” The following will occur:

  1. User will be prompted to enter a name for the saved configuration – in this example, I’ll use www-qa01
  2. The configuration will be saved in S3 as s3://elasticbeanstalk-us-west-2-187376578462/resources/templates/www/www-qa01 (note that the bucket name depends on your region and account ID and the key name depends on the application name and saved configuration name you provided previously)
  3. The configuration will be copied from S3 to saved to local disk as .elasticbeanstalk/saved_configs/www-qa01.cfg.yml (note that the filename depends on the saved configuration name you provided previously)

A before and after screenshot of running the eb config savecommand: Before command – notice that the AWS Console displays no “Saved Configurations”: eb config save - www-qa01 - Before After command – notice that the eb config save command has written a file to disk and that the AWS Console displays a saved configuration named “www-qa01.” eb config save - www-qa01 - After Lastly – a diagram of how the eb config save command works: eb config save - Diagram

eb config delete $environment_name

As an example, if I ran eb config delete www-qa01, then the following would occur:

  1. The www-qa01 saved configuration object will be removed from S3. As an example, if I ran eb config delete www-qa01, then the object s3://elasticbeanstalk-us-west-2-187376578462/resources/templates/www/www-qa01 would be deleted.
  2. As a result of deleting the www-qa01 saved configuration object from S3, the www-qa01 saved configuration will be removed from AWS Console
  3. The YAML configuration file will be removed from .elasticbeanstalk/saved_configs/www-qa01.cfg.yml
eb config get $environment_name

If I ran eb config get www-qa01, then the following would occur:

  1. If exists, a saved configuration file will be copied from S3 (location: s3://elasticbeanstalk-us-west-2-187376578462/resources/templates/www/www-qa01) to local disk at .elasticbeanstalk/saved_configs/www-qa01.cfg.yml
eb config list

If I ran the eb config list, I will execute a “list” against the objects located at s3://elasticbeanstalk-us-west-2-187376578462/resources/templates/www/*.

eb config put $environment_name

Uploads the named saved configuration to an Amazon S3 bucket. As an example, if I ran eb config put www-qa01, then the following would happen:

  1. A file named .elasticbeanstalk/saved_configs/www-qa01.cfg.yml would be uploaded to S3 at the following location: s3://elasticbeanstalk-us-west-2-187376578462/resources/templates/www/.
  2. As a result of a new object being uploaded to S3, the saved configuration will appear in the AWS Console.

You can also upload a specific file (if not in .elasticbeanstalk/saved_configs/www-qa01.cfg.yml) by running the command as follows: eb config put www-qa01 --cfg ~/path/to/saved_config/www-qa02.config.yml where ~/path/to/saved_config/www-qa02.config.yml is a path to a valid Elastic Beanstalk Configuration file.