Operational Efficiency and Optimization for GenAI Applications · Topic 5 of 7

Retrieval Performance and Vector Store Operations

Sharpen and speed up RAG retrieval with hybrid search, reranking, query decomposition and HNSW tuning, and keep vector stores healthy over time.

  • 5 min
In a nutshell
  • Query decomposition, rewriting and metadata filters are query preprocessing.
  • Hybrid search combines keyword and vector scores; in Knowledge Bases set overrideSearchType to HYBRID on a supported store.
  • Reranking improves ordering so fewer chunks reach the model.

What it is

In a RAG application the model can only be as good as the chunks it receives. Retrieval performance means two things: relevance (the right chunks come back) and speed (they come back fast enough for an interactive answer).

Vector store operations keep that true over time. Indexes grow, documents change and bad data slips in, so the store needs monitoring, routine optimization and data quality checks.

User querypreprocessRewritten querydecomposed or filteredhybrid searchVector storekeyword plus vectortop resultsRerankerbest chunksFoundation model

How it works

Query preprocessing

Raw user questions are often vague, multi-part or full of filler.

  • Query decomposition splits a complex question into sub-queries. Bedrock Knowledge Bases supports it as a query transformation (QUERY_DECOMPOSITION) in RetrieveAndGenerate.
  • Query rewriting with a small model turns a conversational follow-up into a standalone search query.
  • Metadata filtering narrows the search by attributes such as department, product or date, with operators like equals, greaterThan and in, combined with andAll or orAll.
  • Implicit metadata filtering lets the knowledge base generate a filter from the query using a metadata schema you describe.

Filters improve both relevance and speed, because the search runs over fewer vectors.

Hybrid search and custom scoring

Search typeMatches onStrong forWeak for
SemanticEmbedding similarityParaphrases, natural questionsExact codes, part numbers, rare names
KeywordText terms (BM25)Exact identifiers and jargonSynonyms and rephrasing
HybridBoth, with combined scoresMixed queries in enterprise contentNeeds score tuning

In Bedrock Knowledge Bases, set overrideSearchType to HYBRID or SEMANTIC in the retrieval configuration. Hybrid search is supported for Amazon RDS, Amazon OpenSearch Serverless and MongoDB vector stores that contain a filterable text field; otherwise the query falls back to semantic search.

In Amazon OpenSearch Service you build hybrid search yourself with a search pipeline that normalizes keyword and vector scores and combines them with weights. Adjusting those weights is the custom scoring the exam mentions: raise the keyword weight when users search by exact identifiers.

Reranking reorders the top results with a reranker model, such as Amazon Rerank or Cohere Rerank on Bedrock. Retrieve a wider candidate set, rerank, then send only the best few chunks to the model. You get better relevance with fewer context tokens.

Exam tip

"Searches for product codes return unrelated documents" points to hybrid search. "The right chunk is retrieved but ranked too low to be used" points to reranking.

Index optimization

Most vector stores use HNSW, an approximate nearest neighbor graph. Its settings trade recall against speed and memory.

SettingWhen appliedHigher value means
mIndex buildMore graph links: better recall, more memory
ef_constructionIndex buildBetter graph quality, slower indexing
ef_searchQuery timeBetter recall, slower queries
  • Quantization of vectors cuts memory and cost for very large indexes, with some recall loss.
  • Embedding dimension matters: smaller vectors search faster and use less memory; test recall before shrinking.
  • Chunk size affects both: smaller chunks are more precise but need more vectors and more results per query.
  • In OpenSearch Service, force merging segments after large bulk loads or many updates can speed up k-NN search.

Vector database query optimization

  • Return only the fields you need, not full documents and vectors.
  • Keep numberOfResults small (Bedrock Knowledge Bases returns up to five by default) and rerank instead of sending many chunks.
  • Filter before the vector search where the engine supports it.
  • Keep the vector store in the same Region as the application and model.

Vector store operational monitoring

Vector storeOpenSearch or AurorametricsCloudWatchlatency, errors, capacityalarmSNSon call teamIngestion jobknowledge base syncstatisticsCloudWatchfailed documentsalarmSNS
WatchWhy
Query latency at p99Users feel tail latency, not averages
Error and rejection ratesSignals capacity limits or bad queries
Capacity, such as OpenSearch Compute Units in Serverless or memory and CPU in managed clustersk-NN graphs are memory heavy
Index size and document countUnexpected growth means duplicates; drops mean failed syncs
Ingestion job statisticsKnowledge base sync reports scanned, indexed and failed documents

Automated index optimization and data quality

Treat the index like a production database with scheduled maintenance.

  • Run knowledge base sync with StartIngestionJob on a schedule or on S3 events, so new and changed documents are indexed and deleted ones are removed.
  • Schedule maintenance with EventBridge Scheduler and Lambda, such as force merges in managed clusters or rebuilding an index when settings change.
  • Validate data before ingestion: drop empty or near-empty files, remove duplicates, check required metadata, and redact sensitive data. AWS Glue Data Quality or a Lambda check can gate the pipeline.
  • Use one embedding model per index. Changing the embedding model means re-embedding the whole corpus, because vectors from different models are not comparable.
  • Keep a retrieval test set and measure recall after each change.

How the exam asks about this

  • "Users search by exact error codes and semantic search misses them" -> hybrid search, with higher keyword weight if needed.
  • "Complex multi-part questions get partial answers" -> query decomposition.
  • "Retrieval recall dropped after a bulk import" -> check ingestion statistics and data quality, then tune or rebuild the index.

Common mistakes

Sending more chunks to fix poor answers. It raises cost and can bury the right chunk. Rerank and send fewer, better chunks.

Mixing embeddings from two models in one index. Similarity scores become meaningless; re-embed everything with one model.

Monitoring only average latency. k-NN queries can have long tails under memory pressure; alarm on p99.

What to remember for the exam

  • Query decomposition, rewriting and metadata filters are query preprocessing.
  • Hybrid search combines keyword and vector scores; in Knowledge Bases set overrideSearchType to HYBRID on a supported store.
  • Reranking improves ordering so fewer chunks reach the model.
  • HNSW: m and ef_construction at build time, ef_search at query time; higher means better recall, slower and more memory.
  • Monitor p99 latency, errors, capacity, index size and ingestion failures; validate data before it is embedded.

Check yourself

  • Which retrieval change helps queries that contain exact product codes?

    Hybrid search, because keyword matching finds exact identifiers that embeddings often miss.

  • How does reranking reduce token cost?

    It moves the most relevant chunks to the top, so you can pass fewer chunks to the model.

  • What must happen when you switch to a new embedding model?

    Re-embed and reindex the whole corpus, because vectors from different models cannot be compared.

Topic 5 of 7 in Operational Efficiency and Optimization for GenAI Applications

Test yourself on Operational Efficiency and Optimization for GenAI Applications

The full AWS AIP-C01 question bank has 827 questions, with timed practice exams and an explanation for every answer.

This study guide is independent and is not affiliated with, endorsed by, or sponsored by AWS. Vendor names, certification names and exam codes are trademarks of their respective owners and are used for identification only.