Service Lifecycle, Toil, Capacity Planning and Autoscaling

What it is

A service is not finished when it launches. It moves through planning, deployment, maintenance and finally retirement, and at every stage it needs enough capacity to meet its SLOs.

Capacity has two parts: making sure the resources can be obtained (quotas, reservations) and making sure the service uses the right amount at each moment (autoscaling).

How it works

Service management across the lifecycle

Toil is manual, repetitive, automatable work with no lasting value that grows with the service, such as restarting a stuck job by hand every night. SRE teams cap toil (Google's guideline is no more than about half of an engineer's time) and spend the rest on engineering that removes it.

Quotas and limits

Quota is not capacity: having quota does not guarantee a zone has free machines at that moment. Check quotas during planning, before a launch or a failover test, and set alerts when usage approaches a quota.

Reservations

A Compute Engine reservation holds capacity for specific machine types in a zone, so VMs can be created when needed, for example for a peak or a regional failover.

• Reservations are billed as running VMs whether used or not; combine them with committed use discounts for a lower rate.
• Automatically consumed reservations are used by any matching VM; specific reservations only by VMs that name them.
• Shared reservations can be used by other projects in the organization that you share them with.

Dynamic Workload Scheduler

Dynamic Workload Scheduler handles scarce accelerators such as GPUs and TPUs. It has two modes:

Capacity from either mode is consumed through Compute Engine, GKE, Vertex AI and Batch; on Compute Engine, Flex Start works through managed instance group resize requests.

Autoscaling

gcloud compute instance-groups managed set-autoscaling web-mig \
  --region=europe-west1 --min-num-replicas=3 --max-num-replicas=20 \
  --target-cpu-utilization=0.6 --cool-down-period=90

gcloud run services update web --region=europe-west1 \
  --min-instances=2 --max-instances=50 --concurrency=80

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: web
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: web
  minReplicas: 3
  maxReplicas: 30
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 60

HPA scales pods; if pods cannot be scheduled, the cluster autoscaler adds nodes. Set realistic resource requests, because both depend on them.

A worked example

A retailer expects ten times normal traffic on one sale day. The SRE team load tests staging to find the per-instance capacity and checks regional vCPU quota, requesting an increase weeks ahead.

They create a reservation for the extra VMs in two zones for the day and add a scaling schedule that raises the managed instance group minimum before the sale. Cloud Run front ends get higher minimum instances to avoid cold starts.

How the exam asks about this

• "A planned event must not fail because VMs cannot be created in the region." Create reservations ahead of time, and confirm quota.
• "Training jobs need GPUs but can start whenever capacity is free." Use Dynamic Workload Scheduler in Flex Start mode.
• "Cloud Run requests are slow after quiet periods." Set minimum instances to keep warm instances ready.