Scalability and Decoupling

What it is

A scalable system adds or removes capacity as demand changes, without a person intervening and without redesign. Scaling out (more instances) is preferred to scaling up (bigger instances), because it also adds redundancy.

Scaling works best when tiers are decoupled. A queue or event stream between producer and consumer lets each side scale at its own pace and absorbs bursts.

How it works

EC2 Auto Scaling policies

An Auto Scaling group (ASG) has a minimum, desired and maximum size, a launch template and scaling policies.

Common target tracking metrics are average CPU utilization and ALBRequestCountPerTarget. For queue consumers, scale on a custom metric such as the backlog per instance (queue length divided by the number of instances), not on CPU alone.

The warm up setting tells the group how long a new instance takes before its metrics count. A cooldown prevents the group from adding or removing instances again too soon.

Making scale out faster

• Pre baked AMIs reduce boot time compared with installing software at launch.
• Warm pools keep pre initialized instances stopped or running ready to join, which shortens scale out.
• Lifecycle hooks pause an instance in Pending:Wait or Terminating:Wait so you can run custom actions, such as registering with a service or draining logs, and then continue.

aws autoscaling complete-lifecycle-action --lifecycle-hook-name drain-hook \
  --auto-scaling-group-name web-asg --lifecycle-action-result CONTINUE \
  --instance-id i-0123456789abcdef0

If the hook times out, the default result applies, which is configurable as continue or abandon.

Mixed fleets and cost

A mixed instances policy combines On Demand instances with Spot Instances across several instance types, which lowers cost. Diversify types and AZs so a Spot interruption in one pool does not stall the group.

Containers and serverless

Scaling data and reads

Databases scale differently from stateless compute.

• RDS: scale up the instance size, add read replicas, and add a cache.
• Aurora: add readers (Aurora Auto Scaling can add and remove them by metric) or use Serverless v2 for capacity that follows load.
• DynamoDB: on demand capacity mode scales with no planning, while provisioned mode uses Application Auto Scaling for the read and write capacity units. DynamoDB Accelerator (DAX) adds an in memory cache for reads.
• ElastiCache: an in memory cache in front of the database cuts repeated reads.
• Amazon CloudFront: serves cached content from edge locations and takes load off the origin.

Decoupling tools

With SQS, a consumer fleet scales on the backlog, and the queue absorbs a spike so nothing is lost while capacity catches up.

Instance refresh

Instance refresh replaces instances in an ASG gradually after a launch template change, with a minimum healthy percentage, which is a scaling group level rolling deployment.