Cost Optimized Storage

What it is

Storage costs grow quietly, because data accumulates and old data is rarely cleaned up. The cost goal is to keep each piece of data on the cheapest storage that still meets its access and durability needs.

The key question is how often the data is read and how fast it must come back. Frequently used data belongs on fast, higher priced storage. Rarely used data belongs on cold, cheap storage that charges more or waits longer to retrieve.

How it works

S3 storage classes

IA stands for infrequent access. The colder classes have a minimum storage duration (30 days for the IA classes, 90 days for Glacier Instant and Flexible Retrieval, and 180 days for Deep Archive), so deleting early still bills for the full period. The IA classes also bill a minimum object size, which makes many tiny objects poor candidates.

Choose Intelligent-Tiering when the access pattern is unpredictable, since it avoids retrieval fees and manual tuning. Choose a fixed IA or Glacier class when the pattern is known.

Lifecycle rules

An S3 lifecycle configuration moves objects between classes and deletes them on a schedule, with no code.

{
  "Rules": [{
    "ID": "age-out-logs",
    "Status": "Enabled",
    "Filter": {"Prefix": "logs/"},
    "Transitions": [
      {"Days": 30, "StorageClass": "STANDARD_IA"},
      {"Days": 90, "StorageClass": "GLACIER"}
    ],
    "Expiration": {"Days": 365}
  }]
}

Rules can also remove old noncurrent versions of versioned objects and clean up incomplete multipart uploads, both of which otherwise keep costing money without being visible.

Other savings in S3

S3 Storage Lens and storage class analysis show where the data is and how it is used. Requester Pays buckets shift the data transfer cost to the person who downloads. Compressing data before storing it also reduces both storage and transfer cost.

EBS savings

• Use gp3 instead of gp2 for most volumes, since it gives a baseline of performance at a lower price per GB and lets you pay for extra IOPS only when needed.
• Use st1 or sc1 HDD volumes for large, sequential, infrequent data.
• Delete volumes that are no longer attached to an instance.
• Right size volumes to what the workload really needs.
• Manage snapshots with Amazon Data Lifecycle Manager, so old ones expire. Snapshots are incremental, but unused ones still add up. The EBS Snapshots Archive tier is cheaper for snapshots kept for a long time.

EFS savings

EFS lifecycle management moves files that are not accessed for a set time to the EFS Infrequent Access or Archive storage classes, which cost less. EFS One Zone stores data in a single AZ and is cheaper than the regional option, so it fits non critical data. Pick the throughput mode that matches real use, because provisioned throughput costs extra.

Storage that grows by itself or needs scaling

Some storage needs no capacity planning. S3 and EFS grow and shrink with your data, and you pay only for what is stored. Other services charge for what you provision, so empty space costs money.

Start with the size you need now and grow when required, instead of provisioning for a distant peak.

Matching to a scenario

For compliance logs that must be kept for years and rarely read, use S3 with a lifecycle rule to Glacier Deep Archive. For a data lake with unknown access, use Intelligent-Tiering. For a shared file system with many cold files, use EFS lifecycle management.