Plan and Manage an Azure AI Solution · Topic 3 of 4
Managing, Monitoring and Securing AI Systems
AI-103 operations: quotas, rate limits and cost, monitoring quality and search health, and securing Foundry with managed identity and private networking.
- 4 min
- Limits are tokens per minute and requests per minute; going over returns 429.
- Retry with exponential backoff; raise quota or use provisioned capacity for steady high load.
- Monitor latency, tokens, errors, quality, drift, safety, grounding and index health.
What it is
Once an AI solution is running, it must stay within quota and budget, keep its quality over time, and be secure. AI-103 tests how to manage rate limits and cost, what to monitor for models, agents and search indexes, and how to lock down access with identity and networking.
How it works
Quotas, rate limits and cost
- Model deployments are limited by tokens per minute (TPM) and requests per minute (RPM). Quota is assigned to a subscription per region, per model and per deployment type.
- When a client goes over the limit, the service returns HTTP 429 (Too Many Requests). Clients should retry with exponential backoff, honoring the retry-after-ms header when it is returned.
- To raise throughput: request more quota, spread load across deployments or regions, or use a provisioned deployment for reserved capacity.
- To lower cost: choose a smaller model, shorten prompts, cache repeated answers, cap output tokens, and use batch deployments for jobs that can wait.
What to monitor
| Area | Examples of what to watch |
|---|---|
| Model performance | Latency, token usage, error rates, throttling (429s) |
| Quality | Relevance, groundedness and coherence scores from evaluations over time |
| Drift | Changes in input patterns or answer quality after a model or data change |
| Safety | Blocked prompts or outputs, jailbreak attempts, flagged content |
| Grounding | Whether answers stay faithful to retrieved sources |
| Data ingestion and search | Indexer failures, document counts, index freshness, retrieval relevance |
Telemetry is usually sent to Application Insights and Azure Monitor, with tracing to follow each request through models and tools.
Security
| Control | What it does |
|---|---|
| Microsoft Entra ID authentication (keyless) | Apps sign in with an identity instead of an API key |
| Managed identity | Gives an Azure resource its own identity, so no secret is stored in code or config |
| Azure RBAC | Grants least-privilege roles on the Foundry resource and project |
| Private networking | Private endpoints and disabled public access keep traffic on the virtual network |
| Key Vault | Stores secrets when a key cannot be avoided |
Foundry roles
Microsoft renamed the Foundry built-in roles; the role IDs and permissions did not change.
| Role | Purpose |
|---|---|
| Foundry Agent Consumer | Interact with agent endpoints only; least privilege for callers of agents |
| Foundry User | Build and test in a project; least-privilege role for developers |
| Foundry Project Manager | Manage projects and conditionally assign Foundry User to others |
| Foundry Account Owner | Full management of the resource and projects; conditionally assign roles |
| Foundry Owner | Full access to manage and build |
The older names were Azure AI User, Azure AI Owner, Azure AI Account Owner and Azure AI Project Manager. A project's managed identity usually needs Foundry User on the Foundry resource, plus data roles on connected resources such as Azure AI Search or Storage.
A worked example: a web app calls an agent and must not store keys. Give the app a managed identity, grant it Foundry Agent Consumer if it only calls agents, and use DefaultAzureCredential in code so it signs in with that identity.
"No keys anywhere" in a question points to managed identity plus Microsoft Entra ID authentication and an RBAC role, not to storing a key in Key Vault.
How the exam asks about this
- "Calls fail with HTTP 429 at peak times." Retry with backoff, and raise quota or add capacity.
- "Which role lets a developer build and test agents with least privilege?" Foundry User.
- "Traffic to the Foundry resource must not cross the public internet." Private endpoints with public network access disabled.
What to remember for the exam
- Limits are tokens per minute and requests per minute; going over returns 429.
- Retry with exponential backoff; raise quota or use provisioned capacity for steady high load.
- Monitor latency, tokens, errors, quality, drift, safety, grounding and index health.
- Prefer keyless authentication with managed identities and RBAC.
- Foundry roles: Agent Consumer, User, Project Manager, Account Owner, Owner.
Do not paste API keys into code or app settings. Use a managed identity, or Key Vault if a key is truly required.
Common mistakes
Retrying immediately in a tight loop after a 429. Back off and wait.
Granting Foundry Owner to an app that only calls agents. Use the least-privilege role.
Monitoring only uptime. Quality, safety and grounding also need to be measured.
Check yourself
What does HTTP 429 mean for a model deployment?
The request exceeded the rate limit or quota; the client should retry with backoff.
How can an app call Foundry without storing any key?
Use a managed identity with Microsoft Entra ID authentication and assign it an RBAC role.
What was Foundry User called before the rename?
Azure AI User.
Topic 3 of 4 in Plan and Manage an Azure AI Solution
Test yourself on Plan and Manage an Azure AI Solution
The full Microsoft AI-103 question bank has 648 questions, with timed practice exams and an explanation for every answer.
This study guide is independent and is not affiliated with, endorsed by, or sponsored by Microsoft. Vendor names, certification names and exam codes are trademarks of their respective owners and are used for identification only.