AI Technologies and Controls · Topic 5 of 7
Privacy, Ethics, Trust and Safety Controls
Privacy-enhancing techniques, fairness and bias, explainability, transparency, risk-based human oversight and trust and safety controls for AI outputs.
- 5 min
- PETs reduce privacy exposure but each has a trade-off; pseudonymized data is still personal data.
- Define fairness for the use case, test before release and monitor in production.
- Transparency means disclosing AI use; explainability means giving reasons.
What it is
AI can be secure and still cause harm. It can expose personal data, treat groups unfairly, give confident but false answers, produce harmful content or make decisions nobody can explain. Privacy, ethical, trust and safety controls protect the people affected by AI and the organization's reputation.
These controls put responsible AI principles, such as those of the OECD and the trustworthy characteristics in the NIST AI RMF, into practice. Human oversight, applied in proportion to risk, ties them together.
How it works
Privacy by design and privacy-enhancing techniques
Privacy starts with purpose limitation, minimization and a lawful basis, assessed in a DPIA where required. Technical privacy-enhancing techniques (PETs) reduce exposure further.
| Technique | What it does | Trade-off |
|---|---|---|
| Anonymization | Removes the link to individuals irreversibly | Hard to achieve fully; reduces data value |
| Pseudonymization | Replaces identifiers with tokens; still personal data in law | Re-identification possible with the key |
| Differential privacy | Adds calibrated noise so results do not reveal any one individual | Lower accuracy as privacy increases |
| Federated learning | Trains across many locations without centralizing raw data | Complex; updates can still leak information |
| Synthetic data | Artificial records resembling real data | Must be tested for leakage and realism |
| Confidential computing | Processes data in protected hardware environments | Platform dependence, cost |
Individuals' rights, such as access, correction and erasure, still apply. Because removing learned data from a model is difficult, privacy decisions are made before training.
Fairness and bias
Bias can enter through historical data, unrepresentative samples, labelling, feature choices or how outputs are used. Controls:
- Define what fair means for the use case with the business owner and legal, since fairness metrics can conflict.
- Test outcomes across relevant groups before release and monitor them in production.
- Mitigate with better data, rebalancing, model changes or human review.
- Document decisions and residual bias in the model card and impact assessment.
Explainability and transparency
| Concept | Meaning | Example control |
|---|---|---|
| Transparency | Being open that AI is used and how | Disclose AI interaction; label AI-generated content; publish model cards |
| Explainability | Being able to give reasons for an output | Explanations for individual decisions; choosing interpretable models for high-risk use |
| Accountability | A named person answers for the outcome | Business owner; documented approvals |
Where AI makes or supports decisions with significant effects on people, explanations and a route to challenge the decision are often legally required as well as ethically expected.
Risk-based human oversight
Human oversight is proportionate: the higher the risk, the closer the human involvement.
| Level | How it works | Fits |
|---|---|---|
| Human in the loop | A person approves each output before it takes effect | High-impact decisions about people, payments, safety |
| Human on the loop | AI acts; a person monitors and can intervene or stop it | High-volume operations with moderate risk |
| Human in command | People set the boundaries and can override or switch off the system | Overall governance of any AI system |
Oversight only works if the person has the competence, authority and time to disagree with the AI. Automation bias, the tendency to accept AI output without question, turns oversight into a rubber stamp. Controls include training, showing confidence and sources, sampling reviewed cases and measuring how often reviewers change outputs.
Oversight of inputs and outputs
Risk-based review covers four qualities named in the outline:
| Quality | What reviewers and controls check |
|---|---|
| Trust and safety | Harmful, abusive or policy-violating content; misuse of the system |
| Quality | Accuracy, relevance, factual grounding; confabulation |
| Explainability | Whether reasons and sources can be given |
| Robustness | Consistent behaviour under unusual or hostile input |
Trust and safety controls
Trust and safety controls protect users and the public from harmful AI behaviour: content policies stating what the AI must not produce, input and output classifiers, abuse detection, user reporting channels, grounding answers in approved sources with citations, and clear disclaimers where output is not advice. An AI ethics review or committee considers uses that are legal but may be harmful or damage trust.
A worked example
A bank uses AI to recommend loan declines. Reviewers approve 99% of recommendations within seconds, and complaints suggest some applicant groups are declined more often.
The security manager reports two problems: automation bias has made oversight ineffective, and fairness is not being monitored. Treatment includes showing reviewers the key factors behind each recommendation, sampling decisions for second review, fairness monitoring across groups with thresholds, and a clear process for applicants to request human reconsideration.
How the exam asks about this
- "What level of human oversight is MOST appropriate for AI that decides on loan applications?" Human in the loop, with competent reviewers who can override.
- "Reviewers accept nearly every AI recommendation. What is the MAIN concern?" Automation bias making oversight ineffective.
- "Which technique allows statistics to be published while limiting what can be learned about any individual?" Differential privacy.
What to remember for the exam
- PETs reduce privacy exposure but each has a trade-off; pseudonymized data is still personal data.
- Define fairness for the use case, test before release and monitor in production.
- Transparency means disclosing AI use; explainability means giving reasons.
- Oversight level follows risk: in the loop, on the loop, in command.
- Automation bias can make human oversight meaningless.
Adding a human reviewer does not make an AI decision safe by itself. If reviewers lack the time, competence or authority to disagree, oversight exists only on paper.
Common mistakes
Treating pseudonymized data as anonymous. It can be re-identified and is still personal data under privacy law.
Measuring fairness once. Outcomes can drift between groups as data and populations change.
Equating transparency with explainability. Telling people AI is used is different from explaining why it reached a result.
Check yourself
What is automation bias?
The tendency of people to accept AI output without enough scrutiny, which weakens human oversight.
When is human in the loop oversight most appropriate?
For high-impact decisions such as those affecting people's rights, finances or safety, where each output should be approved before it takes effect.
What is the trade-off with differential privacy?
Stronger privacy protection requires more noise, which reduces accuracy.
Topic 5 of 7 in AI Technologies and Controls
Test yourself on AI Technologies and Controls
The full ISACA AAISM question bank has 747 questions, with timed practice exams and an explanation for every answer.
This study guide is independent and is not affiliated with, endorsed by, or sponsored by ISACA. Vendor names, certification names and exam codes are trademarks of their respective owners and are used for identification only.