Operational Efficiency and Optimization for GenAI Applications · Topic 5 of 7
Retrieval Performance and Vector Store Operations
Sharpen and speed up RAG retrieval with hybrid search, reranking, query decomposition and HNSW tuning, and keep vector stores healthy over time.
- 5 min
- Query decomposition, rewriting and metadata filters are query preprocessing.
- Hybrid search combines keyword and vector scores; in Knowledge Bases set
overrideSearchTypetoHYBRIDon a supported store. - Reranking improves ordering so fewer chunks reach the model.
What it is
In a RAG application the model can only be as good as the chunks it receives. Retrieval performance means two things: relevance (the right chunks come back) and speed (they come back fast enough for an interactive answer).
Vector store operations keep that true over time. Indexes grow, documents change and bad data slips in, so the store needs monitoring, routine optimization and data quality checks.
How it works
Query preprocessing
Raw user questions are often vague, multi-part or full of filler.
- Query decomposition splits a complex question into sub-queries. Bedrock Knowledge Bases supports it as a query transformation (
QUERY_DECOMPOSITION) in RetrieveAndGenerate. - Query rewriting with a small model turns a conversational follow-up into a standalone search query.
- Metadata filtering narrows the search by attributes such as department, product or date, with operators like
equals,greaterThanandin, combined withandAllororAll. - Implicit metadata filtering lets the knowledge base generate a filter from the query using a metadata schema you describe.
Filters improve both relevance and speed, because the search runs over fewer vectors.
Hybrid search and custom scoring
| Search type | Matches on | Strong for | Weak for |
|---|---|---|---|
| Semantic | Embedding similarity | Paraphrases, natural questions | Exact codes, part numbers, rare names |
| Keyword | Text terms (BM25) | Exact identifiers and jargon | Synonyms and rephrasing |
| Hybrid | Both, with combined scores | Mixed queries in enterprise content | Needs score tuning |
In Bedrock Knowledge Bases, set overrideSearchType to HYBRID or SEMANTIC in the retrieval configuration. Hybrid search is supported for Amazon RDS, Amazon OpenSearch Serverless and MongoDB vector stores that contain a filterable text field; otherwise the query falls back to semantic search.
In Amazon OpenSearch Service you build hybrid search yourself with a search pipeline that normalizes keyword and vector scores and combines them with weights. Adjusting those weights is the custom scoring the exam mentions: raise the keyword weight when users search by exact identifiers.
Reranking reorders the top results with a reranker model, such as Amazon Rerank or Cohere Rerank on Bedrock. Retrieve a wider candidate set, rerank, then send only the best few chunks to the model. You get better relevance with fewer context tokens.
"Searches for product codes return unrelated documents" points to hybrid search. "The right chunk is retrieved but ranked too low to be used" points to reranking.
Index optimization
Most vector stores use HNSW, an approximate nearest neighbor graph. Its settings trade recall against speed and memory.
| Setting | When applied | Higher value means |
|---|---|---|
m | Index build | More graph links: better recall, more memory |
ef_construction | Index build | Better graph quality, slower indexing |
ef_search | Query time | Better recall, slower queries |
- Quantization of vectors cuts memory and cost for very large indexes, with some recall loss.
- Embedding dimension matters: smaller vectors search faster and use less memory; test recall before shrinking.
- Chunk size affects both: smaller chunks are more precise but need more vectors and more results per query.
- In OpenSearch Service, force merging segments after large bulk loads or many updates can speed up k-NN search.
Vector database query optimization
- Return only the fields you need, not full documents and vectors.
- Keep
numberOfResultssmall (Bedrock Knowledge Bases returns up to five by default) and rerank instead of sending many chunks. - Filter before the vector search where the engine supports it.
- Keep the vector store in the same Region as the application and model.
Vector store operational monitoring
| Watch | Why |
|---|---|
| Query latency at p99 | Users feel tail latency, not averages |
| Error and rejection rates | Signals capacity limits or bad queries |
| Capacity, such as OpenSearch Compute Units in Serverless or memory and CPU in managed clusters | k-NN graphs are memory heavy |
| Index size and document count | Unexpected growth means duplicates; drops mean failed syncs |
| Ingestion job statistics | Knowledge base sync reports scanned, indexed and failed documents |
Automated index optimization and data quality
Treat the index like a production database with scheduled maintenance.
- Run knowledge base sync with StartIngestionJob on a schedule or on S3 events, so new and changed documents are indexed and deleted ones are removed.
- Schedule maintenance with EventBridge Scheduler and Lambda, such as force merges in managed clusters or rebuilding an index when settings change.
- Validate data before ingestion: drop empty or near-empty files, remove duplicates, check required metadata, and redact sensitive data. AWS Glue Data Quality or a Lambda check can gate the pipeline.
- Use one embedding model per index. Changing the embedding model means re-embedding the whole corpus, because vectors from different models are not comparable.
- Keep a retrieval test set and measure recall after each change.
How the exam asks about this
- "Users search by exact error codes and semantic search misses them" -> hybrid search, with higher keyword weight if needed.
- "Complex multi-part questions get partial answers" -> query decomposition.
- "Retrieval recall dropped after a bulk import" -> check ingestion statistics and data quality, then tune or rebuild the index.
Common mistakes
Sending more chunks to fix poor answers. It raises cost and can bury the right chunk. Rerank and send fewer, better chunks.
Mixing embeddings from two models in one index. Similarity scores become meaningless; re-embed everything with one model.
Monitoring only average latency. k-NN queries can have long tails under memory pressure; alarm on p99.
What to remember for the exam
- Query decomposition, rewriting and metadata filters are query preprocessing.
- Hybrid search combines keyword and vector scores; in Knowledge Bases set
overrideSearchTypetoHYBRIDon a supported store. - Reranking improves ordering so fewer chunks reach the model.
- HNSW:
mandef_constructionat build time,ef_searchat query time; higher means better recall, slower and more memory. - Monitor p99 latency, errors, capacity, index size and ingestion failures; validate data before it is embedded.
Check yourself
Which retrieval change helps queries that contain exact product codes?
Hybrid search, because keyword matching finds exact identifiers that embeddings often miss.
How does reranking reduce token cost?
It moves the most relevant chunks to the top, so you can pass fewer chunks to the model.
What must happen when you switch to a new embedding model?
Re-embed and reindex the whole corpus, because vectors from different models cannot be compared.
Topic 5 of 7 in Operational Efficiency and Optimization for GenAI Applications
Test yourself on Operational Efficiency and Optimization for GenAI Applications
The full AWS AIP-C01 question bank has 827 questions, with timed practice exams and an explanation for every answer.
This study guide is independent and is not affiliated with, endorsed by, or sponsored by AWS. Vendor names, certification names and exam codes are trademarks of their respective owners and are used for identification only.