Plan and Manage an Azure AI Solution · Topic 4 of 4
Responsible AI for Generative and Agentic Systems
AI-103 responsible AI: guardrails and content filters, prompt shields, safety evaluations, tracing and audit, and governing agents with approvals.
- 4 min
- Includes a diagram
- Six principles: fairness, reliability and safety, privacy and security, inclusiveness, transparency, accountability.
- Guardrails cover harm categories, Prompt Shields, protected material, groundedness and blocklists.
- Blocked prompt: HTTP 400 content filter error. Blocked output: finish reason content_filter.
What it is
Responsible AI means building AI systems that are fair, reliable and safe, private and secure, inclusive, transparent and accountable. Microsoft lists these as its six responsible AI principles. For AI-103 this becomes practical engineering: configuring safety filters, measuring safety with evaluations, keeping audit trails, and limiting what agents can do on their own.
How it works
Microsoft's six principles
| Principle | Meaning |
|---|---|
| Fairness | Treat all people fairly |
| Reliability and safety | Perform reliably and safely |
| Privacy and security | Respect privacy and be secure |
| Inclusiveness | Empower everyone |
| Transparency | Be understandable |
| Accountability | People are accountable for AI systems |
Guardrails and content filters
Foundry applies guardrails (content filtering) to model deployments and agents. A default configuration is applied; you can create custom ones. A guardrail scans user input and output; for agents (preview) it can also scan tool calls and tool responses.
| Control | What it detects |
|---|---|
| Harm categories | Hate, sexual, violence and self-harm content, at configurable severity thresholds |
| Prompt Shields | Jailbreak attempts in user prompts and indirect prompt injection hidden in documents or tool results |
| Protected material | Known copyrighted text or code in outputs |
| Groundedness detection | Answers not supported by the provided sources |
| Blocklists | Custom terms you want blocked |
When a filter blocks a prompt, the API returns an error (HTTP 400 with a content filter code). When it blocks a completion, the response has a finish reason of content_filter. Apps must handle both.
Evaluations and instrumentation
- Quality evaluators score relevance, groundedness, coherence and fluency.
- Safety evaluators score harmful content and jailbreak vulnerability, often with adversarial test data.
- Agent evaluators check things such as whether the agent chose the right tool and completed the task.
- Some model-graded evaluators, such as rubric evaluators, return reasoning with each score, which helps explain why a result passed or failed.
- Run evaluations before release and regularly in production.
Auditing
- Tracing records each step: prompts, model calls, tool calls and responses.
- Provenance metadata records where content came from, such as source documents or content credentials on generated images.
- Approval workflows require a person to approve sensitive actions before they run.
Governing agents
| Control | Purpose |
|---|---|
| Tool access controls | Give the agent only the tools it needs |
| Constraints in instructions | Limit scope, for example "never issue refunds over 100" |
| Human in the loop (approval) | A person approves high-impact actions such as payments or deletions |
| Oversight modes | Choose how autonomous the agent is: suggest only, act with approval, or act alone for low-risk steps |
| Least-privilege identity | The agent's identity can reach only what it must |
A worked example: an agent can read orders and issue refunds. Refunds are high impact, so the refund tool requires human approval, the agent's identity can only call the refund API for its region, and every call is traced for audit.
Indirect prompt injection comes from content the model reads, such as a web page, email or document. Prompt Shields for documents is the control that detects it.
How the exam asks about this
- "A document retrieved by RAG contains hidden instructions." Indirect prompt injection; use Prompt Shields.
- "How does an app know the output was blocked?" The finish reason is content_filter.
- "An agent must not send emails without a manager's OK." Require human approval for that tool.
What to remember for the exam
- Six principles: fairness, reliability and safety, privacy and security, inclusiveness, transparency, accountability.
- Guardrails cover harm categories, Prompt Shields, protected material, groundedness and blocklists.
- Blocked prompt: HTTP 400 content filter error. Blocked output: finish reason content_filter.
- Evaluate quality and safety before release and in production.
- Govern agents with limited tools, constraints, approvals and least-privilege identities.
Turning filters off to fix false positives is the wrong first step. Adjust severity thresholds or use a custom configuration, and evaluate the effect.
Common mistakes
Handling only one kind of block. Prompts and completions are blocked differently.
Treating responsible AI as a policy document only. It is configured, measured and monitored.
Giving agents every available tool. Least privilege applies to tools too.
Check yourself
What does Prompt Shields detect?
Jailbreak attempts in user prompts and indirect prompt injection in documents or other content the model reads.
How can an app tell that a completion was filtered?
The response has a finish reason of content_filter.
Name one way to govern a high-impact agent action.
For example, require human approval before the tool runs.
Topic 4 of 4 in Plan and Manage an Azure AI Solution
Test yourself on Plan and Manage an Azure AI Solution
The full Microsoft AI-103 question bank has 648 questions, with timed practice exams and an explanation for every answer.
This study guide is independent and is not affiliated with, endorsed by, or sponsored by Microsoft. Vendor names, certification names and exam codes are trademarks of their respective owners and are used for identification only.