Plan and Manage an Azure AI Solution · Topic 1 of 4
Choosing Foundry Services and Models
AI-103 service selection: choosing LLMs, small and multimodal models, Foundry Tools, retrieval and indexing methods, and agent memory and tools.
- 4 min
- Pick the smallest model that meets the quality bar; confirm with evaluations.
- Reasoning models suit multistep problems; SLMs suit simple, high-volume work.
- Foundry Tools give prebuilt vision, language, speech, translation and document capabilities.
What it is
Microsoft Foundry is Azure's platform for building AI apps and agents. It brings together a catalog of models, prebuilt AI capabilities called Foundry Tools, the Foundry Agent Service for agents, and tools for evaluation, monitoring and safety.
The first skill AI-103 tests is choosing the right piece for a requirement: which kind of model, which Foundry service, which retrieval method, and which tools and memory an agent needs.
How it works
Choosing a model
| Model type | Strengths | Typical use |
|---|---|---|
| Large language model (LLM) | Broad knowledge, strong reasoning and writing | Complex chat, summarization, agents that plan |
| Reasoning model | Spends extra effort thinking before answering | Multistep math, code, planning, hard analysis |
| Small language model (SLM) | Lower cost and latency, can run at the edge | Simple classification, extraction, high-volume tasks |
| Multimodal model | Accepts text plus images or audio | Describing images, answering questions about screenshots |
| Embedding model | Turns text into vectors | Vector search and RAG |
| Image and video generation models | Create media from prompts | Marketing images, short video clips |
Choose the smallest model that meets the quality bar. A bigger model costs more and responds more slowly; a model that is too small produces poor answers. Evaluations, not guesses, should settle the choice.
Choosing a Foundry service
| Requirement | Good fit |
|---|---|
| Generate text, chat, call tools | A model deployment from the Foundry model catalog |
| Ground answers on company documents | Azure AI Search, or Foundry agent tools such as file search |
| Build an agent that uses tools and keeps a conversation | Foundry Agent Service |
| Extract fields and layout from documents, images, audio or video | Azure Content Understanding in Foundry Tools, or Document Intelligence |
| Detect sentiment, entities or PII in text | Azure Language in Foundry Tools, or a language model with a prompt |
| Translate text | Azure Translator in Foundry Tools, or an LLM translation flow |
| Speech to text or text to speech | Azure Speech in Foundry Tools |
| Block harmful prompts and outputs | Guardrails and content filters, Azure AI Content Safety |
A prebuilt Foundry Tool is often cheaper and more predictable than prompting an LLM for the same narrow task. An LLM is more flexible when the task needs reasoning or a custom output format.
Choosing retrieval and indexing
| Method | How it finds content | Best for |
|---|---|---|
| Keyword (full text) | Matches the words in the query | Exact terms, codes, names |
| Vector | Compares embeddings for meaning | Paraphrased or natural language questions |
| Hybrid | Combines keyword and vector results | Most RAG workloads; a good default |
| Semantic ranking | Reranks results with a language model | Improving the order of the top results |
Agent memory, tools and knowledge
An agent needs three things besides a model:
- Tools to act: function calling, OpenAPI tools, MCP servers, code interpreter, file search, Azure AI Search, web search or Grounding with Bing, computer use, browser automation.
- Knowledge to ground answers: search indexes, uploaded files, SharePoint or Fabric data.
- Memory to keep context: the conversation history and, where needed, longer-term stored facts.
When a question stresses low cost, low latency and a simple task, an SLM or a prebuilt Foundry Tool usually beats a large LLM.
How the exam asks about this
- "Users ask questions in their own words about thousands of PDFs. Which retrieval method?" Hybrid search with vectors.
- "An agent must call an internal REST API described by an OpenAPI spec. Which tool?" An OpenAPI tool.
- "Detect PII in support tickets at high volume and low cost." Azure Language PII detection in Foundry Tools.
What to remember for the exam
- Pick the smallest model that meets the quality bar; confirm with evaluations.
- Reasoning models suit multistep problems; SLMs suit simple, high-volume work.
- Foundry Tools give prebuilt vision, language, speech, translation and document capabilities.
- Hybrid search is a strong default for RAG; semantic ranking improves ordering.
- Agents need tools, knowledge and memory, not just a model.
An embedding model does not generate answers. It produces vectors for search; a chat or reasoning model writes the answer.
Common mistakes
Always choosing the biggest model. It raises cost and latency without always improving results.
Using keyword search alone for natural language questions. Vector or hybrid search finds meaning, not just words.
Prompting an LLM for every narrow task. A prebuilt tool can be cheaper and more consistent.
Check yourself
When is a small language model a better choice than an LLM?
When the task is simple and high volume and low cost or latency matters more than broad reasoning.
What does hybrid search combine?
Keyword (full text) search and vector search.
Name two kinds of tools an agent can use in Foundry Agent Service.
For example, function calling and Azure AI Search, or code interpreter and an OpenAPI tool.
Topic 1 of 4 in Plan and Manage an Azure AI Solution
Test yourself on Plan and Manage an Azure AI Solution
The full Microsoft AI-103 question bank has 648 questions, with timed practice exams and an explanation for every answer.
This study guide is independent and is not affiliated with, endorsed by, or sponsored by Microsoft. Vendor names, certification names and exam codes are trademarks of their respective owners and are used for identification only.