Applications of Foundation Models · Topic 2 of 7
Retrieval Augmented Generation
Understand how RAG grounds a foundation model in your own data using embeddings and vector stores, and how Amazon Bedrock Knowledge Bases supports it.
- 4 min
- Includes a diagram
- RAG retrieves relevant passages and adds them to the prompt so the model answers from them.
- RAG gives access to current and private data without retraining the model.
- RAG helps reduce hallucination and can cite sources.
What it is
Retrieval augmented generation (RAG) lets a foundation model answer with the help of documents that you provide at request time. The system first retrieves relevant passages, then adds them to the prompt so the model can use them.
RAG is popular because it gives the model current, private information without retraining it. It also reduces hallucination, since answers are grounded in real text.
How it works
Preparing the data
Before questions arrive, documents are prepared in an indexing step.
- The documents are split into smaller pieces called chunks.
- An embedding model turns each chunk into a vector, a list of numbers that captures meaning.
- The vectors are stored in a vector database, together with the original text.
Answering a question
When a user asks a question, the question is also turned into a vector. The vector store finds the chunks whose vectors are closest in meaning. These chunks are added to the prompt, and the model writes the answer from them.
Vector database options on AWS
Several AWS services can store and search vectors.
| Service | Note |
|---|---|
| Amazon OpenSearch Service | Search and analytics engine with vector search |
| Amazon Aurora | Relational database, with vector support in the PostgreSQL edition |
| Amazon RDS for PostgreSQL | Relational database, with vector support |
| Amazon Neptune Analytics | Graph analytics engine, with vector search for graph based retrieval |
| Amazon DocumentDB (with MongoDB compatibility) | Document database with vector search |
Pick the vector store that fits what you already run. If your data lives in PostgreSQL, adding vector support there avoids a new system.
Amazon Bedrock Knowledge Bases
Amazon Bedrock Knowledge Bases is a managed RAG feature. You point it at your data source, such as documents in Amazon S3, and choose an embedding model and a vector store. It then handles chunking, embedding, storing and retrieval, and provides the retrieved text to the model, often with source references.
Knowledge Bases works with a defined list of supported vector stores, so check the current list before you design around one. It removes the need to build the whole pipeline yourself, which is why it is the usual exam answer for RAG on Amazon Bedrock.
Why choose RAG
| Need | Why RAG fits |
|---|---|
| Answers from private company data | The documents are fetched at request time |
| Up to date information | Update the documents, not the model |
| Fewer hallucinations | The model is given facts to use |
| Source references | Retrieved passages can be cited |
| Lower cost than retraining | No training job is needed |
RAG business applications
Common uses include customer support over product manuals, employee help desks over policy documents, and search over legal or technical content. Amazon Kendra can also be used as a retriever for enterprise content.
Limits of RAG
RAG answers are only as good as the documents and the retrieval. If the right passage is not found, the model may still guess. Access control matters, since users should only get passages they are allowed to see.
What to remember for the exam
- RAG retrieves relevant passages and adds them to the prompt so the model answers from them.
- RAG gives access to current and private data without retraining the model.
- RAG helps reduce hallucination and can cite sources.
- Embeddings turn text into vectors, and a vector database finds the closest ones.
- Amazon OpenSearch Service, Aurora, RDS for PostgreSQL, Neptune Analytics and DocumentDB can serve as vector stores.
- Amazon Bedrock Knowledge Bases is the managed RAG service.
- Updating the documents is how you update RAG knowledge.
- RAG is cheaper and faster to start than fine tuning.
A scenario that needs answers from the company's own, frequently changing documents points to RAG, not fine tuning.
Common mistakes
Choosing fine tuning to add fresh facts. Fine tuning is slow to update. RAG changes when the documents change.
Thinking RAG changes the model. It does not. It adds information to each prompt.
Forgetting access control. Retrieved passages must respect who may see them.
Believing RAG removes all hallucination. It lowers the risk, but poor retrieval can still lead to wrong answers.
Mixing up the embedding model and the text model. One creates vectors for search, and the other writes the answer.
Check yourself
What problem does RAG solve?
It lets a model answer from current or private data without retraining, and reduces hallucination.
What is stored in a vector database for RAG?
The embeddings (vectors) of document chunks, linked to the original text.
Which AWS feature provides managed RAG on Amazon Bedrock?
Amazon Bedrock Knowledge Bases.
Topic 2 of 7 in Applications of Foundation Models
Test yourself on Applications of Foundation Models
The full AWS AIF-C01 question bank has 849 questions, with timed practice exams and an explanation for every answer.
This study guide is independent and is not affiliated with, endorsed by, or sponsored by AWS. Vendor names, certification names and exam codes are trademarks of their respective owners and are used for identification only.