AI Technologies and Controls · Topic 6 of 7
Designing and Reviewing AI Security Controls
Selecting AI security controls to treat risk to an acceptable level: input, model, output, access and data controls, defence in depth and regular review.
- 5 min
- Controls are chosen from the risk assessment to bring risk within appetite.
- Layer controls across governance, data, model, input, retrieval, output, action and operations.
- Use preventive, detective and corrective controls together.
What it is
AI security controls are the safeguards that bring AI risk down to the level the organization has accepted. They are selected from the risk assessment, designed to fit the AI system's architecture, implemented, tested and reviewed regularly, exactly as for any other control, but against AI-specific threats.
No single control is reliable on its own against attacks such as prompt injection, so AI controls are layered. The aim is not a perfect model; it is a system in which a fooled model cannot cause unacceptable harm.
How it works
Controls by layer
| Layer | Example controls |
|---|---|
| Governance | Use case approval, risk tiering, policy, roles, exception process |
| Data | Classification, minimization, provenance, integrity checks, access control on datasets |
| Model | Registry with approvals, integrity checks on weights, encryption, access control, safety tuning |
| Input | Authentication, input validation and length limits, prompt injection detection, content filters |
| Context and retrieval | Separation of trusted instructions from untrusted content, permission-aware retrieval |
| Output | Content and sensitive data filtering, validation and encoding before onward use, grounding and citation checks |
| Action | Least privilege tools, human approval for high-impact actions, sandboxing |
| Operational | Rate and cost limits, logging, monitoring, incident playbooks, rollback |
Preventive, detective and corrective
| Type | AI examples |
|---|---|
| Preventive | Least privilege for agents, data minimization, input filters, approval gates |
| Detective | Logging and alerting on injection patterns, drift monitoring, output scanning, fairness monitoring |
| Corrective | Rollback to a known-good model, kill switch, revoking keys, removing poisoned data |
| Deterrent and directive | Acceptable use policy, user warnings, terms of service |
| Compensating | Human review where a technical control is not yet available |
A balanced design uses all types. Preventive controls fail against new techniques, so detective and corrective controls must be ready.
Guardrails in context
Guardrails are controls that screen or constrain AI inputs and outputs, often using classifiers or rules. They are useful but imperfect: they can be bypassed, and they can block legitimate use. Their performance is measured, tuned to the use case and backed by architecture controls such as least privilege and output validation.
Selecting controls
Selection considers:
- Risk reduction: which threats from the assessment the control treats, and how much.
- Cost and effort: to implement and to operate, compared with the risk reduced.
- Effect on the business: latency, false positives that block users, impact on usefulness.
- Fit with existing controls: reuse enterprise identity, DLP, logging and key management.
- Regulatory requirements: controls required by law or contract, such as logging and human oversight for high-risk AI.
Control ownership and documentation
Each control has an owner, a documented objective, the risks it treats and how its effectiveness is tested. Controls are mapped to framework and regulatory requirements, for example ISO/IEC 42001 reference controls or EU AI Act duties, so one control can provide evidence for several requirements.
Testing control effectiveness
A control that exists is not necessarily a control that works. Testing includes red team exercises aimed at the control, automated evaluation sets that measure how often guardrails block attacks and how often they block legitimate requests, configuration reviews, and checks that alerts reach responders.
Regular review
AI controls are reviewed on a schedule and when triggers occur: a new model version, new tools or data sources, new threat techniques, incidents, audit findings or changes in regulation. A control tuned for one model may behave differently on another, so model changes always prompt a control check.
A worked example
A customer assistant is about to launch. The risk assessment lists prompt injection, data leakage, harmful output and cost abuse.
The manager approves a layered set: authentication and rate limits at input; an injection classifier; retrieval restricted to public help content; output filtering for personal data and harmful content; no tools that change data; logging into security monitoring; and a kill switch held by the service owner. Testing shows the injection classifier blocks many attacks but not all, which is acceptable because the assistant has nothing sensitive to reveal and no actions to take. The service owner accepts the documented residual risk.
How the exam asks about this
- "What should determine which AI security controls are implemented?" The AI risk assessment and the organization's risk appetite.
- "A guardrail blocks many legitimate customer requests. What should the security manager do FIRST?" Measure and tune it with the business owner, balancing risk reduction and usability.
- "The organization switches to a new model provider. What should happen to existing AI controls?" Retest and review them, because controls may behave differently with the new model.
What to remember for the exam
- Controls are chosen from the risk assessment to bring risk within appetite.
- Layer controls across governance, data, model, input, retrieval, output, action and operations.
- Use preventive, detective and corrective controls together.
- Guardrails are useful but bypassable; back them with architecture controls.
- Test effectiveness and review at model, threat and regulatory changes.
Do not accept a vendor's claim that a guardrail "stops prompt injection". Measure it in your own use case and design so that a bypass cannot cause unacceptable harm.
Common mistakes
Relying on one control. A single filter or system prompt instruction is bypassable; defence in depth is required.
Ignoring false positives. Controls that block legitimate work get disabled or bypassed by users.
Never revisiting controls. A model change, new tool or new technique can make a working control ineffective overnight.
Check yourself
Why are AI controls layered?
Because no single control reliably stops attacks such as prompt injection, so several layers limit the chance and impact of failure.
Give an example of a corrective AI control.
Rolling back to a known-good model version, using a kill switch, revoking keys or removing poisoned data.
When should AI controls be reviewed outside the normal schedule?
When a model, tool or data source changes, a new threat appears, an incident occurs, audit findings arise or regulation changes.
Topic 6 of 7 in AI Technologies and Controls
Test yourself on AI Technologies and Controls
The full ISACA AAISM question bank has 747 questions, with timed practice exams and an explanation for every answer.
This study guide is independent and is not affiliated with, endorsed by, or sponsored by ISACA. Vendor names, certification names and exam codes are trademarks of their respective owners and are used for identification only.