Data Store Management · Topic 1 of 7
Choosing a Data Store
Match access pattern, scale, latency and cost to the right AWS data store, comparing S3, Redshift, DynamoDB, RDS, Aurora, OpenSearch and other databases.
- 5 min
- S3 is the data lake store. Redshift is the warehouse. Athena queries S3 with SQL and no servers.
- DynamoDB is for key value access at massive scale with low latency, not for ad hoc joins.
- RDS and Aurora are for relational OLTP. Redshift is for OLAP.
What it is
AWS offers many data stores, and each is built for a different access pattern. Choosing one means asking how the data will be read and written, how fast answers must arrive, how big the data will get and what it may cost.
The exam rewards purpose built thinking. A key value lookup, an ad hoc analytical query and a full text search each point to a different service.
How it works
The main stores
| Store | Type | Choose it for |
|---|---|---|
| Amazon S3 | Object storage | Data lake, raw and curated files, cheap and durable storage of any size |
| Amazon Redshift | Columnar data warehouse | Complex SQL analytics and joins across large structured datasets |
| Amazon Athena | Serverless SQL over S3 | Ad hoc queries on files, with no warehouse to run |
| Amazon DynamoDB | Key value and document NoSQL | Very high scale, low latency lookups by key |
| Amazon RDS and Aurora | Relational (OLTP) | Transactions, referential integrity, existing SQL applications |
| Amazon OpenSearch Service | Search and analytics | Full text search, log analytics, dashboards |
| Amazon DocumentDB | Document, MongoDB compatible | Moving MongoDB style workloads |
| Amazon Keyspaces | Wide column, Apache Cassandra compatible | Cassandra workloads without managing servers |
| Amazon Neptune | Graph | Highly connected data, relationships and traversals |
| Amazon Timestream | Time series | Metrics and IoT data indexed by time |
| Amazon MemoryDB and ElastiCache | In memory | Caching and microsecond reads |
OLTP (online transaction processing) means many small reads and writes. OLAP (online analytical processing) means fewer, larger queries that scan and aggregate a lot of data.
Storage services behind the databases
Not every need is a database. Amazon S3 is object storage, Amazon EBS (Elastic Block Store) is block storage attached to one EC2 instance, and Amazon EFS (Elastic File System) is a shared network file system for Linux. Amazon FSx offers managed file systems for specific needs, such as Windows file shares or Lustre for high performance computing.
For a data lake the answer is almost always S3, because it decouples storage from compute, scales without planning and works with Athena, Glue, EMR and Redshift Spectrum.
Decision factors
Work through these in order.
- Data shape: structured tables, key value items, documents, graphs or files.
- Access pattern: point lookups by key, joins and aggregations, or text search.
- Latency: milliseconds for an application, or seconds to minutes for a report.
- Scale and throughput: how much data and how many requests.
- Consistency and transactions: do you need ACID (atomicity, consistency, isolation, durability) transactions across rows?
- Cost model: pay for provisioned capacity, pay per query, or pay for storage only.
- Operations: managed or serverless options reduce effort.
Warehouse, lake and lakehouse
A data warehouse such as Redshift stores structured, modelled data for fast SQL. A data lake in S3 stores raw and processed data in many formats at low cost. A lakehouse combines both, usually by keeping data in S3 and querying it from Redshift Spectrum or Athena, with a shared catalog.
Redshift Spectrum lets a Redshift query read external tables in S3 without loading them. This is the usual answer when a question asks to query cold data in S3 together with warehouse tables.
Migration and hybrid needs
Consider how data gets into the store.
Redshift loads from S3 with COPY. DynamoDB can export to and import from S3. DMS can move data between engines.
If a company must keep a specific engine such as Oracle or PostgreSQL, RDS or Aurora preserves compatibility.
Cost and performance trade offs
S3 is the cheapest place to keep large volumes, but a raw file scan is slower than a tuned warehouse. Redshift is faster for repeated complex queries and many users. DynamoDB on demand capacity suits unpredictable traffic, while provisioned capacity with auto scaling can cost less for steady load.
What to remember for the exam
- S3 is the data lake store. Redshift is the warehouse. Athena queries S3 with SQL and no servers.
- DynamoDB is for key value access at massive scale with low latency, not for ad hoc joins.
- RDS and Aurora are for relational OLTP. Redshift is for OLAP.
- OpenSearch Service is for search and log analytics.
- Neptune is for graph, Timestream for time series, Keyspaces for Cassandra, DocumentDB for MongoDB compatibility.
- Spectrum lets Redshift query S3 data in place.
- ElastiCache and MemoryDB serve in memory needs.
- Decouple storage and compute when workloads vary, with S3 as the base.
Read the verbs in the question. "Look up by key" suggests DynamoDB, "aggregate across years of data" suggests Redshift or Athena, and "search text" suggests OpenSearch.
Common mistakes
Using DynamoDB for analytics. It has no joins and scanning large tables is slow and costly. Export to S3 and use Athena or Redshift.
Using Redshift as an OLTP database. It is built for large scans, not for thousands of small transactions per second.
Choosing a warehouse when files in S3 would do. For occasional queries, Athena avoids paying for a running cluster.
Forgetting compatibility. A lift and shift of a MongoDB or Cassandra application is simplest on DocumentDB or Keyspaces.
Do not pick a store from the data volume alone. The access pattern usually decides the answer.
Check yourself
Which store fits low latency lookups by key at very large scale?
Amazon DynamoDB.
How can Redshift query cold data that stays in S3?
Through Redshift Spectrum, which reads external tables in S3.
Which service is built for full text search and log analytics?
Amazon OpenSearch Service.
Topic 1 of 7 in Data Store Management
Test yourself on Data Store Management
The full AWS DEA-C01 question bank has 845 questions, with timed practice exams and an explanation for every answer.
This study guide is independent and is not affiliated with, endorsed by, or sponsored by AWS. Vendor names, certification names and exam codes are trademarks of their respective owners and are used for identification only.