Model Components and Configurations · Topic 2 of 2
Choosing and Deploying a Model
AI-901 model choice and deployment: the Foundry catalog, selection criteria, standard, provisioned and batch deployments, quotas, and key parameters.
- 5 min
- The catalog offers many models. Benchmarks and the playground help you compare them.
- A model must be deployed to get an endpoint and a deployment name.
- Standard is pay per token. Provisioned reserves capacity. Batch is for large jobs that can wait.
What it is
Microsoft Foundry offers a model catalog with models from Microsoft, OpenAI and other providers. Choosing well means matching the model to the task, then deploying it so an application can call it, and finally setting the parameters that control its behaviour.
How it works
Choosing a model
Start from the task and work through a few criteria.
| Criterion | Question to ask |
|---|---|
| Capability and modality | Does it handle the input and output you need: text, images, audio, embeddings? |
| Quality | Does it give good enough answers on your own test prompts? |
| Cost | What does it cost per token, or per image? |
| Speed | What latency does the application need? |
| Context window | Is it large enough for your prompts and replies? |
| Safety and region | Does it meet your safety needs and run in the region you need? |
Match model type to task.
| Task | Model type |
|---|---|
| Draft a reply to a customer email | Chat completion model |
| Rank support articles by meaning | Embedding model |
| Describe the contents of a product photo | Multimodal model |
| Work through a multi step puzzle | Reasoning model |
| Create a picture from a description | Image generation model |
| Fast, low cost text tasks | Small language model such as Phi |
Foundry helps with the decision. The catalog lists models from several providers. Model benchmarks compare quality, cost and throughput side by side. The playground lets you try a prompt on a candidate model before you commit.
Deployment
A catalog model must be deployed before an application can call it. Deployment creates an endpoint and a deployment name, and your code uses the deployment name to say which model to use. You do not need to retrain or download the model.
Deployment options differ in how you pay and what performance you get.
| Option | Idea | Good for |
|---|---|---|
| Standard | Pay for the tokens you use | Most applications, variable use |
| Provisioned throughput | Reserve capacity for predictable speed | High volume, steady workloads |
| Batch | Submit large jobs to run later at lower cost | Big jobs that can wait |
Each deployment has a quota, measured in tokens per minute and requests per minute. If an app goes over it, the service returns HTTP status 429, and the fix is to wait and retry with exponential backoff, or to raise the quota. Rotating the key or recreating the deployment does not change the limit.
Configuration parameters
Parameters change how the model picks tokens and when it stops.
| Parameter | What it does | Typical use |
|---|---|---|
| temperature | Lower is more focused and consistent. Higher is more varied | Low for factual answers, higher for creative ones |
| top_p | Samples only from the most likely tokens that add up to this share | An alternative way to control variety |
| max_tokens | Upper limit on tokens in the reply | Raise it if answers stop mid sentence |
| frequency_penalty | Discourages tokens that already appeared often | Reduce repetition |
| presence_penalty | Encourages new topics | More varied content |
| stop sequences | Text that ends the output | Cut the answer at a marker |
Microsoft advises changing temperature or top p, but not both together, and testing the same prompts after each change so you can see what the change did.
Two limits can be confused. max_tokens caps the reply. The context window caps the input plus the reply together.
A worked scenario
A team builds a summariser. In the catalog they compare three chat models on cost and quality benchmarks, and test their own sample documents in the playground. They pick the one that gives good summaries at the lowest cost, and deploy it with a deployment name summariser. Summaries keep ending mid sentence, so they raise max_tokens. Results vary between runs, so they lower the temperature. During a busy period the app starts getting HTTP 429 responses, so the code now waits and retries with exponential backoff.
Matching a symptom to a setting
| Symptom | Likely change |
|---|---|
| Output stops mid sentence | Increase max_tokens |
| Answers are too random | Lower temperature |
| The same phrase repeats | Raise frequency_penalty |
| HTTP 429 | Retry with backoff, or raise the quota |
| Request rejected, prompt plus max_tokens is above the window | Shorten the prompt or lower max_tokens |
What to remember for the exam
- The catalog offers many models. Benchmarks and the playground help you compare them.
- A model must be deployed to get an endpoint and a deployment name.
- Standard is pay per token. Provisioned reserves capacity. Batch is for large jobs that can wait.
- Lower temperature makes output more consistent. Higher max_tokens allows longer replies.
- HTTP 429 means a rate limit or quota was exceeded.
Change temperature or top p, not both, and compare outputs on the same test prompts after each change.
How the exam asks about this
- "Summaries stop mid sentence. Which parameter should be increased?" max_tokens.
- "What must be done to a catalog model before an app can call it?" Deploy it.
- "The call returns HTTP 429. What should the app do?" Wait and retry with exponential backoff.
- "Which model type suits a multi step scheduling puzzle?" A reasoning model.
Common mistakes
Retraining a model just to call it. Deployment is enough.
Changing temperature and top p together. You cannot tell which one caused the change.
Mixing up max_tokens and the context window. One limits the reply, and the other limits input plus reply.
Answering a 429 by changing the key. The key is fine. The rate limit was exceeded.
Check yourself
What does deploying a catalog model give you?
An endpoint and a deployment name that applications can call.
Which parameter makes responses more consistent when lowered?
Temperature.
What does HTTP 429 mean?
The rate limit or quota was exceeded, so wait and retry.
Topic 2 of 2 in Model Components and Configurations
Test yourself on Model Components and Configurations
The full Microsoft AI-901 question bank has 511 questions, with timed practice exams and an explanation for every answer.
This study guide is independent and is not affiliated with, endorsed by, or sponsored by Microsoft. Vendor names, certification names and exam codes are trademarks of their respective owners and are used for identification only.