Optimize Generative AI Systems and Model Performance · Topic 1 of 3
Retrieval Tuning and Embedding Models
AI-300 optimization: tune chunk size, overlap, similarity thresholds and retrieval strategy for RAG, and select or fine-tune embedding models for a domain.
- 5 min
- Smaller chunks give precision, larger give context; overlap protects boundaries.
- Vector search always returns k results; a threshold lets you reject weak ones.
- Same embedding model for indexing and querying;
dimensionsmust match the index.
What it is
In retrieval-augmented generation (RAG) the model answers from passages found by a search step. If the right passage is not retrieved, no prompt can save the answer. Retrieval quality depends on how documents are split into chunks, how chunks are turned into vectors by an embedding model, and how results are filtered and ranked. Optimizing means changing one of these and measuring the effect.
How it works
Chunk size and overlap
| Choice | Effect |
|---|---|
| Small chunks | Precise matches, but may lose surrounding meaning and need more chunks per answer |
| Large chunks | More context per hit, but diluted embeddings, more tokens and more noise |
| Overlap | Repeats text across neighbouring chunks so a sentence cut at a boundary is still found; too much overlap duplicates content and wastes tokens |
| Structure aware splitting | Split on headings, paragraphs or tables rather than a fixed count so chunks stay meaningful |
Microsoft's chunking guidance gives a starting point of roughly 512 tokens with about 25 percent overlap, but the right value depends on your documents, so test several. In Azure AI Search the Text Split skill sets maximumPageLength and pageOverlapLength; a layout aware skill can split documents by structure. Add useful metadata (title, section, date) to each chunk for filtering and for citations.
Similarity thresholds and top k
A vector query returns the k nearest chunks, which always returns something even if nothing is relevant.
| Setting | Effect |
|---|---|
| k / top | How many chunks are retrieved. Higher k raises recall but adds noise and tokens |
| Similarity threshold | A minimum score; chunks below it are dropped, so the app can say "not found" instead of answering from weak matches |
| Reranker score cut off | After semantic reranking, scores (0 to 4) can be used to drop weak results |
| Filters | Restrict by metadata (product, language, date) before ranking |
Tune the threshold on your evaluation set: too high loses valid answers (low recall); too low lets in irrelevant text (low precision, ungrounded answers). Remember score scales differ between vector similarity, hybrid ranking and reranking, so set thresholds per method.
For the vector index itself, Azure AI Search supports approximate search with HNSW and exact search with exhaustive KNN; HNSW has parameters (such as m, efConstruction, efSearch) that trade speed for recall.
Retrieval strategies
| Strategy | Use when |
|---|---|
| Vector only | Meaning matters more than exact words |
| Keyword (full text) only | Exact identifiers, codes, names dominate |
| Hybrid | Mixed needs, the usual default (next page) |
| Hybrid plus semantic reranker | Best ranking quality for natural language questions |
| Query rewriting / multi query | Short or ambiguous questions; split or rephrase before searching |
Select an embedding model
| Factor | Consider |
|---|---|
| Quality on your domain | Test with your own queries, using retrieval metrics (recall at k, mean reciprocal rank, NDCG) |
| Dimensions | More dimensions can capture more but cost storage and speed; some models let you shorten output with a dimensions setting |
| Max input length | Must fit your chunk size |
| Languages | Multilingual needs |
| Cost and throughput | Embedding millions of chunks is a real bill |
The Azure OpenAI family includes text-embedding-3-small (1,536 dimensions by default), text-embedding-3-large (3,072) and text-embedding-ada-002 (1,536). The index vector field's dimensions must match the model. Queries and documents must use the same model; changing the model means re-embedding and reindexing everything.
Fine-tune or adapt embeddings
If general embeddings retrieve poorly on specialist vocabulary (legal, medical, internal product names), you have options in cost order:
- Improve chunking, metadata and add hybrid search.
- Add a reranker.
- Choose a different or larger embedding model.
- Fine-tune an open embedding model (for example a sentence transformer) with pairs of queries and matching passages, plus hard negatives, using a training job on Azure Machine Learning compute, then deploy it to a managed endpoint.
Azure OpenAI embedding models are not fine-tuned in the way chat models are, so domain adaptation of embeddings usually means an open model that you host. Always evaluate the tuned model on a held out query set and compare against the baseline before reindexing production.
A worked example
A product documentation bot misses answers about error codes. The team measures recall at 5 on 200 labelled questions (0.62). Moving from 2,000 character chunks with no overlap to heading based chunks with overlap lifts it to 0.74. A similarity threshold of 0.8 stops weak matches from reaching the model, reducing ungrounded answers. They keep the same embedding model because a larger one only added 1 point.
How the exam asks about this
- "Answers cite irrelevant passages when nothing matches." Raise the similarity threshold so weak matches are dropped.
- "A sentence split across chunks is never retrieved." Add chunk overlap.
- "Embeddings were switched to a new model and results broke." Documents and queries must use the same model; re-index.
What to remember for the exam
- Smaller chunks give precision, larger give context; overlap protects boundaries.
- Vector search always returns k results; a threshold lets you reject weak ones.
- Same embedding model for indexing and querying;
dimensionsmust match the index. - Measure with recall at k, MRR or NDCG on a labelled query set.
- Domain adaptation of embeddings means fine-tuning an open model you host.
Change one retrieval setting at a time. If you change chunking and the model together, you cannot tell which helped.
Common mistakes
Maximizing k. More chunks add noise and tokens, and can lower answer quality.
Mixing embedding models in one index. Vectors from different models are not comparable.
Tuning on a few examples. Use a labelled evaluation set.
Check yourself
What does a similarity threshold do?
It drops chunks scoring below a minimum, so weak matches do not reach the model.
Why add overlap between chunks?
So content at a boundary still appears whole in at least one chunk.
What must you do after changing the embedding model?
Re-embed and reindex all documents, and use the same model for queries.
Topic 1 of 3 in Optimize Generative AI Systems and Model Performance
Test yourself on Optimize Generative AI Systems and Model Performance
The full Microsoft AI-300 question bank has 650 questions, with timed practice exams and an explanation for every answer.
This study guide is independent and is not affiliated with, endorsed by, or sponsored by Microsoft. Vendor names, certification names and exam codes are trademarks of their respective owners and are used for identification only.