Principles of Responsible AI · Topic 1 of 2
Fairness, Reliability and Safety
AI-901 responsible AI: fairness measured per group, counterfactual testing, reliability and safety, content filters, red teaming and mitigation layers.
- 5 min
- Includes a diagram
- Fairness means equitable treatment. Compare results for each group, because overall accuracy can hide a gap.
- Counterfactual testing changes only a demographic detail and checks the output.
- Reliability and safety mean consistent, dependable, harm resistant behaviour.
What it is
Two of Microsoft's six principles are about how an AI system behaves. Fairness means the system treats all people and groups equitably, so that similar people are not affected in very different ways. Reliability and safety mean the system performs consistently, copes with unexpected input and does not cause harm.
An AI system learns from data and produces answers that look confident, so both principles need deliberate checks. They do not happen by default.
How it works
Where unfairness comes from
Bias enters an AI solution in several places.
| Source | Example |
|---|---|
| Training data | A hiring model trained mostly on past hires from one group |
| Labels | Human reviewers apply a rating unevenly to different groups |
| Model design | A feature that stands in for a protected trait, such as a postcode |
| Use | A tool built for one population is applied to another |
A system can look accurate overall while being unfair to a group, because an overall average hides the difference.
Measuring and reducing unfairness
The key idea is to compare results for each group, not only in total.
- A credit model may be 92 percent accurate for two applicant groups, yet reject qualified applicants of one group more often. Comparing the false negative rate for each group exposes that.
- Counterfactual testing changes only a demographic detail, such as a name or a pronoun, in otherwise identical inputs, and checks whether the output changes.
- Representative data includes enough examples of every group the system will serve.
- Fairness metrics and tools, such as the open source Fairlearn toolkit, help assess and reduce differences between groups.
- Human review of borderline or high impact decisions catches errors the model makes.
Reliability and safety
A reliable system works as intended under normal and unusual conditions. A safe system avoids causing harm. Typical measures are:
- Test widely, including edge cases, unusual inputs and different languages.
- Set content filters. In Microsoft Foundry, guardrails screen prompts and responses for categories such as hate, sexual content, violence and self harm, and a stricter setting blocks more.
- Red team the system. Testers deliberately try to make it fail or misbehave, so weaknesses are found before users find them.
- Evaluate and monitor. Built in evaluators score responses, for example for groundedness or for unfair language, and monitoring watches the system after release.
- Plan for failure. Provide fallbacks, limits and a way for a person to take over.
Risks specific to generative AI
| Risk | What it is | Reduction |
|---|---|---|
| Hallucination | The model states something false with confidence | Ground answers in supplied data and tell the model to say it does not know |
| Harmful output | Offensive or dangerous content | Content filters and safety instructions |
| Prompt injection and jailbreaks | A user or a document tricks the model into ignoring its rules | Prompt shields, input checks and limited permissions |
Four layers of mitigation
Microsoft describes harm reduction in four layers, from the model outward.
- The model. Choose a model that suits the task and has strong safety behaviour.
- The safety system. Add content filters and detection around the model.
- The system message and grounding. Give clear instructions and supply trusted data.
- The user experience. Show the limits, ask for confirmation, and let users give feedback.
A worked scenario
A bank tests a loan approval model. Accuracy is high overall, but the false negative rate is noticeably higher for one group, so qualified applicants in that group are rejected more often. The team rebalances the training data, runs counterfactual tests that swap names, and sends borderline cases to a human reviewer. For the chat assistant that explains decisions, they switch on content filters, red team it for jailbreaks, and tell it to say "I don't know" when the supplied policy text does not contain the answer.
What to remember for the exam
- Fairness means equitable treatment. Compare results for each group, because overall accuracy can hide a gap.
- Counterfactual testing changes only a demographic detail and checks the output.
- Reliability and safety mean consistent, dependable, harm resistant behaviour.
- Content filters, red teaming, evaluation and monitoring support reliability and safety.
- Grounding and "say I don't know" instructions reduce hallucinations.
A scenario about unequal outcomes for groups is fairness. A scenario about failures, harmful output or unexpected input is reliability and safety.
How the exam asks about this
- "Which measurement exposes that qualified applicants in one group are rejected more often?" The false negative rate for each group.
- "Which technique changes only a name in otherwise identical inputs?" Counterfactual testing.
- "Reduce harmful content in responses." Content filters.
- "Deliberately try to break the system before release." Red teaming.
Common mistakes
Trusting overall accuracy. A high average can hide a group that is treated badly.
Mixing up fairness and inclusiveness. Fairness is about equal outcomes. Inclusiveness is about designing for people of all abilities.
Thinking a filter removes every risk. Filters are one layer, and testing, grounding and human oversight are still needed.
Believing a model that sounds sure is correct. Fluent output can be false.
Check yourself
How do you check whether a model treats groups differently?
Compare its results, such as the false negative rate, for each group instead of only overall.
What is red teaming?
Deliberately trying to make an AI system fail or misbehave so weaknesses are found before release.
Which action reduces hallucinations?
Grounding the model in supplied data and telling it to say it does not know when the data lacks the answer.
Topic 1 of 2 in Principles of Responsible AI
Test yourself on Principles of Responsible AI
The full Microsoft AI-901 question bank has 511 questions, with timed practice exams and an explanation for every answer.
This study guide is independent and is not affiliated with, endorsed by, or sponsored by Microsoft. Vendor names, certification names and exam codes are trademarks of their respective owners and are used for identification only.