Foundation Model Integration, Data Management, and Compliance · Topic 7 of 9

Advanced Retrieval, Query Handling and Access Patterns

Improve RAG relevance with hybrid search and Bedrock rerankers, rewrite and decompose queries, and expose retrieval with function calling and MCP.

  • 6 min
  • Includes a diagram
In a nutshell
  • Hybrid search combines keyword and vector scores; OpenSearch normalizes scores in a search pipeline.
  • Knowledge base queries can use semantic or hybrid search where the vector store supports it.
  • Bedrock reranker models rescore candidates; retrieve wide, rerank, keep the top few.

What it is

Basic RAG embeds the question, takes the top few nearest chunks and sends them to the model. That fails on exact terms such as product codes, on vague or multi part questions, and when the best chunk ranks fifth instead of first.

Advanced retrieval fixes the search itself, the query going into it and the way the model reaches it. The goal is that the chunks in the prompt are the right ones.

User questionrewrite or splitBetter queriesexpansion, decompositionhybrid searchCandidateskeyword plus vectorrerankTop chunksmost relevantaugmentFMgrounded answer

How it works

Search architectures

Hybrid search with reranking: a query runs keyword (BM25) and semantic (k-NN) search in OpenSearch, a normalization processor combines the scores, a reranker model such as Amazon Rerank or Cohere Rerank rescores dozens of candidates and only the top few chunks go into the prompt.

TechniqueWhat it doesFixes
Semantic searchNearest neighbor search on embeddingsDifferent words with the same meaning
Keyword searchLexical match, such as BM25 in OpenSearchExact terms: IDs, error codes, names
Hybrid searchRuns both and combines the scoresQueries that need meaning and exact terms
RerankingA reranker model rescores the candidate chunks against the queryRelevant chunks ranked too low

In Amazon OpenSearch Service, a hybrid query combines a keyword query with a neural or k-NN query. A search pipeline with a normalization processor puts the two score scales on a common range before combining them.

In Amazon Bedrock Knowledge Bases, the Retrieve and RetrieveAndGenerate APIs let you set the search type to semantic or hybrid, for vector stores that support hybrid search such as OpenSearch Serverless.

Amazon Bedrock reranker models, such as Amazon Rerank and Cohere Rerank, score each candidate passage against the query. You can call the Rerank API directly or add a reranking configuration to a knowledge base query. A common pattern retrieves a wide set, such as dozens of chunks, then keeps only the top few after reranking.

QueryRetrieveKnowledge basemany candidatesrerankReranker modelrelevance scoreskeep top fewPrompt context
Exam tip

If the scenario says users search for part numbers or error codes and get unrelated results, add keyword matching through hybrid search. If the right chunk is retrieved but ranked too low, add a reranker.

Query handling

User questions are often short, vague or compound. Improve them before searching.

TechniqueHowAWS tool
Query expansionAdd synonyms, related terms or rewrites, then search with eachAn FM on Amazon Bedrock
Query decompositionSplit a compound question into sub questions, retrieve for each, then combineLambda calling an FM, or Bedrock Knowledge Bases query decomposition
Query transformationMulti step rewrite, routing and retrieval with branches and retriesAWS Step Functions
Conversational rewriteTurn a follow up such as "and for Europe?" into a full question using historyAn FM with the chat history

Bedrock Knowledge Bases supports query decomposition as a query transformation option in RetrieveAndGenerate. It breaks a complex query into smaller queries, retrieves for each and uses the combined results.

Compound questiondecomposeLambdaFM callsub question oneRetrieveresults ALambdaFM callsub question twoRetrieveresults BmergeFMfinal answer

Step Functions suits longer transformations: classify the query, choose a knowledge base, run parallel retrievals with a Parallel or Map state, then merge and rerank.

Consistent access mechanisms

Applications and agents should reach retrieval the same way every time, not through custom code in every app.

Function calling (tool use). In the Converse API you describe tools in toolConfig, each with a name, description and JSON input schema. When the model wants a tool, the response has a stop reason of tool_use and the arguments. Your code runs the search, returns a toolResult, and the model writes the answer.

UserquestionFMwith tool spectool_use search_docsAppruns vector searchtoolResult chunksFManswerUser

Model Context Protocol (MCP). MCP is an open protocol that standardizes how AI applications discover and call tools and data sources. An MCP server exposes vector search as a tool; any MCP client, such as an agent, can call it without custom integration. Amazon Bedrock AgentCore Gateway can turn existing APIs and Lambda functions into MCP compatible tools.

Standardized API patterns. A shared retrieval service, for example behind API Gateway or the knowledge base Retrieve API, gives every application the same request, filters and response shape.

MechanismBest when
Function callingOne application lets the model decide when to search
MCP serverMany agents and clients must share the same retrieval tools
Shared retrieval APIMany applications need identical retrieval behavior and filters

How the exam asks about this

  • "Searches for SKU codes return semantically similar but wrong products" -> Hybrid search that combines keyword and vector scores.
  • "The correct passage is retrieved but often not in the top three" -> Retrieve more candidates and apply a Bedrock reranker model.
  • "Questions compare two policies and answers miss one side" -> Query decomposition into sub questions.
  • "Several agent frameworks must use the same document search tool" -> Expose it through an MCP server.

Common mistakes

Pure vector search for exact identifiers. Embeddings blur codes and names. Hybrid search restores exact matching.

Sending many raw chunks instead of reranking. More context raises cost and can bury the right passage. Rerank and keep the best.

Searching with the raw follow up question. "What about last year?" has no meaning alone. Rewrite it with the conversation history first.

Letting the model run the tool. In function calling, the model only requests the tool; your application executes it and returns the result.

Watch out

Query expansion that adds too many terms can drift off topic and lower precision. Measure retrieval quality before and after.

What to remember for the exam

  • Hybrid search combines keyword and vector scores; OpenSearch normalizes scores in a search pipeline.
  • Knowledge base queries can use semantic or hybrid search where the vector store supports it.
  • Bedrock reranker models rescore candidates; retrieve wide, rerank, keep the top few.
  • Expansion widens recall, decomposition handles compound questions, Step Functions orchestrates multi step transformation.
  • Function calling: toolConfig with JSON schema, tool_use stop reason, app returns toolResult.
  • MCP standardizes tool access for many clients; AgentCore Gateway exposes APIs and Lambda functions as MCP tools.

Check yourself

  • When does a reranker help most?

    When relevant chunks are retrieved but ranked too low, because the reranker rescores candidates directly against the query.

  • How does an FM use a vector search tool through the Converse API?

    The model returns a tool_use request with arguments, the application runs the search and sends back a toolResult, and the model answers.

  • Why use MCP for retrieval tools?

    It gives many agents and clients one standard way to discover and call the same search tools without custom integrations.

Topic 7 of 9 in Foundation Model Integration, Data Management, and Compliance

Test yourself on Foundation Model Integration, Data Management, and Compliance

The full AWS AIP-C01 question bank has 827 questions, with timed practice exams and an explanation for every answer.

This study guide is independent and is not affiliated with, endorsed by, or sponsored by AWS. Vendor names, certification names and exam codes are trademarks of their respective owners and are used for identification only.