Criteria for Evaluating Deployment Systems

Criteria for Evaluating Deployment Systems

My goal with this blog post is to provide criteria for evaluating deployment systems – and through this evaluation to either ensure that either the first build of a deployment system or an upgrade of a deployment system meets a company’s needs. The evaluation criteria are broken down into two sections:

  1. User-Facing: attributes of a deployment system that that an application’s user might experience (downtime)
  2. Engineering facing: attributes of a deployment system that Engineering organization will experience (efficiency, ease of deployments).

The deployment system attributes are listed from “Less Mature” (left-most column) to “Most Mature” (right-hand column).

User-Facing (External) Attributes:

The diagram below compares deployment systems from a user’s perspective. Availability is the key measurement. The user only experiences a deployment when an outage or error occurs.

Deployment Systems - User Attributes

  • Availability: can the application function normally during a deployment?
  • Recoverability: if a deployment fails, can the application continue to function normally?

Engineering-Facing (Internal) Attributes:

The diagram below compares deployment systems from an Engineering employee’s perspective. Availability, efficiency, security, scale and transparency are the key measurements.

Deployment Systems - Engineering Attributes

  • Auditing: can the question “who deployed what, where and when” be answered?
  • Dependency Support: does the deployment system also manage resources that the application depends on? Example: does the deployment system manage database schema? Does the deployment system update content delivery network files?
  • Environment Support: how much effort is required to add a new environment? Example: can a “QA2” or “Load Testing” environment be added easily?
  • Re-use of Tooling: can the deployment system be easily re-used to support other applications?
  • Reproducibility: can a given deployment be reproduced? Example: can you give me last week’s release on last week’s Linux server?
  • Rollback: in the event of a failure can the application be rolled back? Can any previous version of the application be deployed?
  • Scale: can the deployment system deploy to any number of nodes?
  • Target Users and Ease of Use: is the deployment system used by both Engineering and Operations or by Operations only?
  • Transparency: does the deployment system integrate with existing tools? Example: are deployment events integrated with an organization’s dashboards?

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s