Foundation Model Integration, Data Management, and Compliance · Topic 7 of 9
Advanced Retrieval, Query Handling and Access Patterns
Improve RAG relevance with hybrid search and Bedrock rerankers, rewrite and decompose queries, and expose retrieval with function calling and MCP.
- 6 min
- Includes a diagram
- Hybrid search combines keyword and vector scores; OpenSearch normalizes scores in a search pipeline.
- Knowledge base queries can use semantic or hybrid search where the vector store supports it.
- Bedrock reranker models rescore candidates; retrieve wide, rerank, keep the top few.
What it is
Basic RAG embeds the question, takes the top few nearest chunks and sends them to the model. That fails on exact terms such as product codes, on vague or multi part questions, and when the best chunk ranks fifth instead of first.
Advanced retrieval fixes the search itself, the query going into it and the way the model reaches it. The goal is that the chunks in the prompt are the right ones.
How it works
Search architectures

| Technique | What it does | Fixes |
|---|---|---|
| Semantic search | Nearest neighbor search on embeddings | Different words with the same meaning |
| Keyword search | Lexical match, such as BM25 in OpenSearch | Exact terms: IDs, error codes, names |
| Hybrid search | Runs both and combines the scores | Queries that need meaning and exact terms |
| Reranking | A reranker model rescores the candidate chunks against the query | Relevant chunks ranked too low |
In Amazon OpenSearch Service, a hybrid query combines a keyword query with a neural or k-NN query. A search pipeline with a normalization processor puts the two score scales on a common range before combining them.
In Amazon Bedrock Knowledge Bases, the Retrieve and RetrieveAndGenerate APIs let you set the search type to semantic or hybrid, for vector stores that support hybrid search such as OpenSearch Serverless.
Amazon Bedrock reranker models, such as Amazon Rerank and Cohere Rerank, score each candidate passage against the query. You can call the Rerank API directly or add a reranking configuration to a knowledge base query. A common pattern retrieves a wide set, such as dozens of chunks, then keeps only the top few after reranking.
If the scenario says users search for part numbers or error codes and get unrelated results, add keyword matching through hybrid search. If the right chunk is retrieved but ranked too low, add a reranker.
Query handling
User questions are often short, vague or compound. Improve them before searching.
| Technique | How | AWS tool |
|---|---|---|
| Query expansion | Add synonyms, related terms or rewrites, then search with each | An FM on Amazon Bedrock |
| Query decomposition | Split a compound question into sub questions, retrieve for each, then combine | Lambda calling an FM, or Bedrock Knowledge Bases query decomposition |
| Query transformation | Multi step rewrite, routing and retrieval with branches and retries | AWS Step Functions |
| Conversational rewrite | Turn a follow up such as "and for Europe?" into a full question using history | An FM with the chat history |
Bedrock Knowledge Bases supports query decomposition as a query transformation option in RetrieveAndGenerate. It breaks a complex query into smaller queries, retrieves for each and uses the combined results.
Step Functions suits longer transformations: classify the query, choose a knowledge base, run parallel retrievals with a Parallel or Map state, then merge and rerank.
Consistent access mechanisms
Applications and agents should reach retrieval the same way every time, not through custom code in every app.
Function calling (tool use). In the Converse API you describe tools in toolConfig, each with a name, description and JSON input schema. When the model wants a tool, the response has a stop reason of tool_use and the arguments. Your code runs the search, returns a toolResult, and the model writes the answer.
Model Context Protocol (MCP). MCP is an open protocol that standardizes how AI applications discover and call tools and data sources. An MCP server exposes vector search as a tool; any MCP client, such as an agent, can call it without custom integration. Amazon Bedrock AgentCore Gateway can turn existing APIs and Lambda functions into MCP compatible tools.
Standardized API patterns. A shared retrieval service, for example behind API Gateway or the knowledge base Retrieve API, gives every application the same request, filters and response shape.
| Mechanism | Best when |
|---|---|
| Function calling | One application lets the model decide when to search |
| MCP server | Many agents and clients must share the same retrieval tools |
| Shared retrieval API | Many applications need identical retrieval behavior and filters |
How the exam asks about this
- "Searches for SKU codes return semantically similar but wrong products" -> Hybrid search that combines keyword and vector scores.
- "The correct passage is retrieved but often not in the top three" -> Retrieve more candidates and apply a Bedrock reranker model.
- "Questions compare two policies and answers miss one side" -> Query decomposition into sub questions.
- "Several agent frameworks must use the same document search tool" -> Expose it through an MCP server.
Common mistakes
Pure vector search for exact identifiers. Embeddings blur codes and names. Hybrid search restores exact matching.
Sending many raw chunks instead of reranking. More context raises cost and can bury the right passage. Rerank and keep the best.
Searching with the raw follow up question. "What about last year?" has no meaning alone. Rewrite it with the conversation history first.
Letting the model run the tool. In function calling, the model only requests the tool; your application executes it and returns the result.
Query expansion that adds too many terms can drift off topic and lower precision. Measure retrieval quality before and after.
What to remember for the exam
- Hybrid search combines keyword and vector scores; OpenSearch normalizes scores in a search pipeline.
- Knowledge base queries can use semantic or hybrid search where the vector store supports it.
- Bedrock reranker models rescore candidates; retrieve wide, rerank, keep the top few.
- Expansion widens recall, decomposition handles compound questions, Step Functions orchestrates multi step transformation.
- Function calling: toolConfig with JSON schema,
tool_usestop reason, app returnstoolResult. - MCP standardizes tool access for many clients; AgentCore Gateway exposes APIs and Lambda functions as MCP tools.
Check yourself
When does a reranker help most?
When relevant chunks are retrieved but ranked too low, because the reranker rescores candidates directly against the query.
How does an FM use a vector search tool through the Converse API?
The model returns a tool_use request with arguments, the application runs the search and sends back a toolResult, and the model answers.
Why use MCP for retrieval tools?
It gives many agents and clients one standard way to discover and call the same search tools without custom integrations.
Topic 7 of 9 in Foundation Model Integration, Data Management, and Compliance
Test yourself on Foundation Model Integration, Data Management, and Compliance
The full AWS AIP-C01 question bank has 827 questions, with timed practice exams and an explanation for every answer.
This study guide is independent and is not affiliated with, endorsed by, or sponsored by AWS. Vendor names, certification names and exam codes are trademarks of their respective owners and are used for identification only.