Chapter 2 of 3 · ISACA AAIA
AI Operations
AI data, development lifecycle, change and configuration, supervision, testing, threats and AI incident response for auditors.
- 8topics
- 41 minfast track
- 2diagrams and exhibits
- 46%of the exam
Topics in this chapter
Take them in order, or jump to the one you need.
- 1Data Management Specific to AIAuditing AI data: input requirements, appropriateness, representativeness and bias, lineage, labelling quality, data splits…5 min
- 2AI Solution Development LifecycleAuditing the AI lifecycle: design, development, validation, deployment, monitoring and decommissioning, MLOps, model…5 min
- 3Change and Configuration Management for AIAuditing AI change and configuration management: retraining as change, prompt and parameter changes, vendor model updates…Diagram5 min
- 4Supervision of AI Outputs, Impacts and DecisionsAuditing AI supervision: human oversight models, output monitoring, KPIs and KRIs, drift and fairness metrics, explainability…Diagram5 min
- 5Testing Techniques for AI SolutionsAI testing for auditors: holdout and cross-validation, confusion matrix metrics, bias, robustness, explainability, red teaming…6 min
- 6Threats and Vulnerabilities Specific to AIAI threats for auditors at concept level: poisoning, evasion, extraction, inference, prompt injection, OWASP Top 10 for LLM…5 min
- 7AI Threat, Vulnerability and Access ProgramsEvaluating AI threat and vulnerability management and identity and access management: threat models, scanning, AI bill of…5 min
- 8Incident Response Management Specific to AIAuditing AI incident and problem management: AI incident types, detection, containment by rollback or kill switch, root cause…5 min
About this chapter
What this chapter covers
AI Operations is 46% of the ISACA AAIA exam, the largest domain. It tests whether you can assess an organization's risk profile, weigh the rewards and consequences of AI implementations, and judge whether the organization is operationally ready to run AI safely.
The domain covers data management specific to AI (2A), AI solution development methodologies and lifecycle (2B), change management specific to AI (2C), supervision of AI solutions, including outputs, impacts and decisions (2D), testing techniques for AI solutions (2E), threats and vulnerabilities specific to AI (2F), and incident response management specific to AI (2G).
The exam has 90 multiple choice questions, and candidates must hold CISA or an approved audit or accounting designation.
Questions are scenarios from the IT auditor's viewpoint. Expect to choose the BEST evidence that a control operated, the FIRST thing to check when a model misbehaves, or the MOST significant gap in a lifecycle, change or monitoring process. Familiar IT general controls appear again, with the AI twist: data and models change behaviour, not just code.
How the topics fit together
Data feeds the lifecycle, testing gates each release, and change management controls every later update to code, data or model. Supervision watches outputs and decisions in production, while threat management and incident response deal with what goes wrong, by accident or by attack.
In AI Operations questions, retraining a model is a change. Any option that lets a new model version, new training data or a new prompt reach production without testing and approval should be treated as a control weakness.
How to use these pages
Each topic page has the same shape, so you can skim for what you need.
- What it is: the idea in plain words.
- How it works: the mechanism, with tables and examples.
- What to remember for the exam: the facts to memorise and how questions are phrased.
- Common mistakes: the classic traps.
- Check yourself: three questions to test your understanding.
Ready for more practice?
The full ISACA AAIA question bank has 750 questions, with timed practice exams and an explanation for every answer.
This study guide is independent and is not affiliated with, endorsed by, or sponsored by ISACA. Vendor names, certification names and exam codes are trademarks of their respective owners and are used for identification only.