Design Resilient Architectures · Topic 3 of 5
Load Balancing and Auto Scaling
Scale horizontally with Elastic Load Balancing and EC2 Auto Scaling, and choose between ALB, NLB and Gateway Load Balancer and among scaling policies.
- 5 min
- Includes a diagram
- ALB works at layer 7 and routes on path or host. NLB works at layer 4 and offers static IPs and very high performance.
- GWLB inserts virtual security appliances into the traffic path.
- Place the load balancer and the ASG across at least two AZs for availability.
What it is
Vertical scaling makes one server bigger. Horizontal scaling adds more servers and shares the work, which is the approach the exam favors because it also improves availability.
Two services do most of the work. Elastic Load Balancing (ELB) spreads incoming traffic across healthy targets. Amazon EC2 Auto Scaling adds and removes instances automatically to match demand.
How it works
Choosing a load balancer
| Type | Layer | Best for | Notable features |
|---|---|---|---|
| Application Load Balancer (ALB) | 7, HTTP and HTTPS | Web apps, microservices, containers | Path and host based routing, Lambda targets, authentication with Cognito, WebSocket |
| Network Load Balancer (NLB) | 4, TCP, UDP and TLS | Very high performance, low latency, non HTTP protocols | Static IP address per Availability Zone (AZ), preserves the client source IP |
| Gateway Load Balancer (GWLB) | 3 | Inserting third party security and inspection appliances | Transparent traffic routing to a fleet of virtual appliances |
Targets can be EC2 instances, IP addresses, containers and (for ALB) Lambda functions. A load balancer sends traffic only to targets that pass health checks. If a target fails the check, traffic goes to the others.
Useful features include these.
- Cross zone load balancing: spreads traffic evenly across targets in all enabled AZs. It is on by default for an ALB and off by default for an NLB.
- Sticky sessions: keep a user on the same target, which hides a stateless design problem, so use them with care.
- Connection draining (deregistration delay): lets in flight requests finish before a target is removed.
- TLS termination: the load balancer decrypts HTTPS using a certificate from AWS Certificate Manager (ACM), so instances do not handle certificates.
EC2 Auto Scaling groups
An Auto Scaling group (ASG) manages a fleet built from a launch template, which says which image, instance type, security groups and roles to use. You set the minimum, desired and maximum size, and choose subnets in several AZs so the fleet is spread out.
The group also replaces instances that fail health checks. When you attach a load balancer, the group can use the load balancer health check, which is better at catching a failed application than the default instance check.
Scaling policies
| Policy | How it works | Use it when |
|---|---|---|
| Target tracking | Keeps a metric near a target, such as average CPU at a chosen percentage | Simplest, and the usual best answer |
| Step scaling | Adds or removes different amounts at different alarm thresholds | You need finer control over big and small changes |
| Scheduled scaling | Changes capacity at set times | Demand follows a known calendar, such as a daily peak |
| Predictive scaling | Uses machine learning on past load to scale ahead of time | Regular patterns that repeat daily or weekly |
Metrics can be CPU, network traffic, or requests per target from the load balancer. Another well known pattern is to scale on the SQS backlog per instance, using a custom metric built from queue length.
A scale in event can end instances that are doing work. Scale in protection and lifecycle hooks give you a way to finish work before the instance goes away.
Scaling beyond EC2
Many services scale through Application Auto Scaling, including DynamoDB capacity, ECS service task counts, Aurora replicas and Lambda provisioned concurrency. Aurora Serverless and DynamoDB on demand scale without a policy that you must write.
Placing a load balancer in front
For a global or very spiky audience, CloudFront can sit in front of the load balancer and cache static content close to users, which reduces the load on the origin servers.
When the question wants the most responsive and cost efficient scaling with the least tuning, pick target tracking. When it describes a predictable peak, such as "every weekday at 9 AM", scheduled scaling fits.
What to remember for the exam
- ALB works at layer 7 and routes on path or host. NLB works at layer 4 and offers static IPs and very high performance.
- GWLB inserts virtual security appliances into the traffic path.
- Place the load balancer and the ASG across at least two AZs for availability.
- Target tracking is the simplest scaling policy, and scheduled scaling fits predictable demand.
- The ASG replaces unhealthy instances, and ELB health checks catch application failures.
- Cross zone load balancing is on by default for ALB and off by default for NLB.
- Scaling out is limited by the maximum size, and by account and AZ capacity.
- Decouple with SQS and scale workers on queue depth for uneven workloads.
Common mistakes
Choosing an NLB for path based routing. Only an ALB inspects HTTP paths and hosts.
Running the whole group in one AZ. An AZ outage then takes the application down. Spread the group across AZs.
Relying on sticky sessions to solve state. Sessions on one instance are lost when it fails. Keep state outside the instance.
Scaling on CPU for an I/O bound or queue driven app. The metric may not move when load rises. Use requests per target or queue depth.
A very low minimum size saves money, but a new instance takes time to start and pass health checks, so a sudden spike can hurt users before capacity arrives.
Check yourself
Which load balancer routes traffic by URL path?
The Application Load Balancer, which works at layer 7.
Which scaling policy keeps average CPU near a chosen value?
Target tracking scaling.
Why spread an Auto Scaling group across several AZs?
So the application keeps running if one AZ fails, since the group launches instances in the remaining AZs.
Topic 3 of 5 in Design Resilient Architectures
Test yourself on Design Resilient Architectures
The full AWS SAA-C03 question bank has 823 questions, with timed practice exams and an explanation for every answer.
This study guide is independent and is not affiliated with, endorsed by, or sponsored by AWS. Vendor names, certification names and exam codes are trademarks of their respective owners and are used for identification only.