AI Governance and Program Management · Topic 6 of 6
AI Incident Response and Business Continuity
Handling AI security incidents: detection, evidence, containment, notification, escalation, eradication and recovery, plus AI in continuity and DR plans.
- 6 min
- Includes a diagram
- Extend the existing incident plan with AI categories, playbooks and roles.
- Preserve model version, prompts, outputs, retrieved data and tool calls as evidence.
- Containment can mean disabling AI, removing agent permissions or isolating data sources.
What it is
An AI incident is an event in which an AI system causes or threatens harm: leaked data, manipulated outputs, a poisoned model, a stolen model, an agent taking unauthorized actions, or outputs that are unsafe, discriminatory or illegal. Many AI incidents are not classic breaches, so the incident process must recognize them.
Business continuity and disaster recovery planning must also account for AI. When a business process depends on a model or an external AI service, its failure or withdrawal is a disruption that needs a plan.
How it works
Preparing for AI incidents
The existing incident response plan is extended, not replaced. Preparation includes:
- AI incident categories and severity criteria, including harm to people, not only data loss.
- Playbooks for likely scenarios such as prompt injection leading to data exposure, poisoned training or retrieval data, model theft, harmful output and agent misuse.
- Roles that include model owners, data scientists, legal, privacy and communications.
- Logging sufficient to reconstruct events: prompts, outputs, retrieved sources, tool calls, model version and configuration.
- Kill switches and rollback options agreed in advance with business owners.
Investigating and documenting
AI investigations need evidence that ordinary logs may not hold. The team preserves the model version and configuration, the system prompt, relevant prompts and outputs, retrieved documents, agent tool calls and training or retrieval data that may be involved. Chain of custody applies as usual.
Root cause can be hard to pin down because model behaviour is probabilistic. The investigation asks whether the cause was the input, the retrieved data, the model, the integration or a missing control.
Containment, eradication and recovery
| Phase | AI-specific options |
|---|---|
| Containment | Disable the AI feature or switch to a fallback; remove the agent's tool permissions; take a poisoned retrieval source offline; tighten rate limits; block an abusive account; revoke exposed keys |
| Eradication | Remove poisoned records from datasets or vector stores; retrain or roll back to a clean model; fix the integration flaw; update guardrails |
| Recovery | Restore a known-good model version from the registry; validate it before returning to service; monitor closely |
Containment choices are a business decision informed by security. Switching off a customer-facing AI service has a cost, so the plan defines in advance who can make that call and under what conditions.
Notification, escalation and reporting
AI incidents follow the organization's escalation matrix, with clear thresholds for informing senior management. External obligations come from regulation and contracts:
| Source | Example duty |
|---|---|
| Privacy law | Under GDPR, personal data breaches are reported to the supervisory authority within 72 hours of becoming aware, where required |
| EU AI Act | Providers of high-risk AI report serious incidents to market surveillance authorities; a deployer that finds one informs the provider first, then the relevant authorities |
| Contracts | Notify customers or partners within agreed times; vendors must notify the organization |
| Sector rules | Financial, health and critical infrastructure reporting regimes |
Legal counsel confirms which duties apply; the security manager ensures the facts needed for reporting are captured quickly.
AI in business continuity and disaster recovery
The business impact analysis identifies processes that depend on AI and what happens when the AI is unavailable or untrustworthy.
| Continuity concern | Planning response |
|---|---|
| External model or API outage | Fallback provider, degraded mode or manual process |
| Vendor withdraws or changes a model | Contract notice periods; tested alternative models |
| Model corrupted or poisoned | Backups of model versions, training data and pipelines in the registry |
| Long retraining time | Include retraining and validation time in recovery time estimates |
| Loss of key staff | Documented models and pipelines; cross-trained team |
Recovery objectives for AI must count the time to validate a restored or retrained model, not only the time to redeploy it. A model restored quickly but untested can repeat the original harm.
A worked example
A customer chatbot starts revealing other customers' order details. Investigation shows the retrieval index was built with a filter missing, so the model retrieved records the user was not entitled to see.
The team switches the chatbot to a fallback that answers only general questions, preserves logs and the index, and informs legal, who assess breach notification. The index is rebuilt with permission-aware retrieval and tested before the chatbot returns. Lessons learned add a pre-release access test to the deployment procedure.
How the exam asks about this
- "An AI model is producing harmful outputs to customers. What should be done FIRST?" Contain, such as disabling the feature or switching to a fallback, according to the incident plan.
- "What evidence is MOST important to preserve after a suspected prompt injection incident?" Prompts, outputs, retrieved content, tool calls and the model version and configuration.
- "Which BEST supports recovery from a poisoned model?" A registry with versioned, known-good models and training data backups.
What to remember for the exam
- Extend the existing incident plan with AI categories, playbooks and roles.
- Preserve model version, prompts, outputs, retrieved data and tool calls as evidence.
- Containment can mean disabling AI, removing agent permissions or isolating data sources.
- Notification duties come from privacy law, the EU AI Act, contracts and sector rules.
- BIA and recovery plans must include AI dependencies, fallbacks and validation time.
Restoring a model without validating it can bring the incident straight back. Recovery is complete only when the restored or retrained model passes its tests.
Common mistakes
Treating harmful output as a quality issue, not an incident. Discriminatory, unsafe or data-leaking output is an incident with possible legal duties.
Not logging enough to investigate. Without prompts, outputs and model versions, the team cannot reconstruct what happened or prove what did not.
Ignoring third-party AI in continuity plans. An external model API is a critical dependency that can fail or change without warning.
Check yourself
Why must AI incident playbooks include harm to people, not only data loss?
Because AI incidents include unsafe, discriminatory or manipulated outputs that cause harm and may trigger legal duties without any data breach.
What is a common containment step for an agent misusing its tools?
Remove or restrict the agent's tool permissions, or disable the agent, while evidence is preserved.
Why must recovery time for AI include validation time?
A restored or retrained model must be tested before use, or it may repeat the harm.
Topic 6 of 6 in AI Governance and Program Management
Test yourself on AI Governance and Program Management
The full ISACA AAISM question bank has 747 questions, with timed practice exams and an explanation for every answer.
This study guide is independent and is not affiliated with, endorsed by, or sponsored by ISACA. Vendor names, certification names and exam codes are trademarks of their respective owners and are used for identification only.