Securing AI Systems · Topic 1 of 7
AI Threat-Modeling Resources
Use OWASP Top 10 for LLM and ML Security Top 10, MIT AI Risk Repository, MITRE ATLAS, CVE AI Working Group and STRIDE to threat model AI systems.
- 6 min
- Includes a diagram
- OWASP LLM Top 10 covers applications built on language models; OWASP ML Security Top 10 covers attacks on models and training.
- The MIT AI Risk Repository is a broad, sorted database of risks from many sources, using causal and domain taxonomies.
- MITRE ATLAS is the AI counterpart of ATT&CK, with tactics, techniques, mitigations and case studies.
What it is
Threat modeling is the structured process of asking what can go wrong with a system and what to do about it. For artificial intelligence (AI) systems, several public resources list the threats so you do not have to invent them.
The exam gives a scenario and asks which resource fits. A top 10 list ranks risks, a repository catalogues them, a knowledge base describes attacker behaviour, and a framework gives you a method.
How it works
Choosing the right resource
| Resource | What it is | Use it when you need |
|---|---|---|
| Open Worldwide Application Security Project (OWASP) Top 10: LLM Top 10 | Top risks in applications built on large language models (LLMs) | Reviewing a chatbot, copilot or agent |
| OWASP Top 10: Machine Learning (ML) Security Top 10 | Top attacks on ML models and training pipelines | Reviewing a classifier or any trained model |
| Massachusetts Institute of Technology (MIT) AI Risk Repository | A living database of AI risks gathered from many published sources | A broad risk inventory, including societal and safety risks |
| MITRE Adversarial Threat Landscape for Artificial-Intelligence Systems (ATLAS) | Adversary tactics, techniques, mitigations and case studies against AI | Mapping attacker behaviour and planning red team tests |
| Common Vulnerabilities and Exposures (CVE) AI Working Group | A CVE Program group on recording AI vulnerabilities | Tracking specific flaws in AI products |
OWASP LLM Top 10
The objectives use the names from the list's first editions: prompt injection, insecure output handling, training data poisoning, model denial of service (DoS), supply chain vulnerabilities, sensitive information disclosure, insecure plug-in design, excessive agency, overreliance and model theft.
The 2025 edition renamed and regrouped some entries. Insecure output handling became improper output handling, model DoS became unbounded consumption, poisoning became data and model poisoning, and new entries cover system prompt leakage, vector and embedding weaknesses and misinformation. Model theft is now covered under unbounded consumption, and overreliance under misinformation.
OWASP ML Security Top 10
Its entries are input manipulation, data poisoning, model inversion, membership inference, model theft, AI supply chain attacks, transfer learning attacks, model skewing, output integrity attacks and model poisoning.
Most attack names in objective 2.6 come from these two lists. LLM list entries concern the application around a language model, while ML list entries concern the model and its data.
MIT AI Risk Repository
The repository sorts risks with a causal taxonomy: who causes the risk (human or AI), whether it is intentional, and whether it arises before or after deployment. A domain taxonomy groups them into seven domains: discrimination and toxicity, privacy and security, misinformation, malicious actors and misuse, human-computer interaction, socioeconomic and environmental harms, and AI system safety, failures and limitations.
Use it when a scenario asks for a wide view of risk rather than technical attacks.
MITRE ATLAS
ATLAS is modelled on MITRE ATT&CK (Adversarial Tactics, Techniques, and Common Knowledge): tactics are the attacker's goals and techniques are the methods, with identifiers starting AML.T. It also lists mitigations and case studies of real attacks.
Most tactics match ATT&CK, such as Initial Access and Exfiltration. Two are specific to AI: AI Model Access, gaining access to the model through an application programming interface (API), a product or the model files, and AI Attack Staging, preparing the attack, for example by building a proxy model or crafting adversarial inputs.
CVE AI Working Group
The CVE Program assigns identifiers to specific vulnerabilities in specific products. Its AI Working Group addresses what counts as a vulnerability in an AI system and how such records should be described.
A CVE record suits a concrete flaw, such as a model-serving library that runs code from a loaded file, not a general weakness such as hallucination.
Threat-modeling frameworks
STRIDE sorts threats into six categories and works well on a data flow diagram.
| STRIDE category | AI example |
|---|---|
| Spoofing | A deepfake voice impersonates an executive to an AI assistant |
| Tampering | Poisoned training data or an altered model file |
| Repudiation | Agent actions with no log of which prompt caused them |
| Information disclosure | The model reveals training data or its system prompt |
| Denial of service | Huge prompts exhaust the model's capacity or budget |
| Elevation of privilege | Prompt injection makes an agent use tools the user cannot |
PASTA (Process for Attack Simulation and Threat Analysis) is a risk-centred method that starts from business objectives. Attack trees break one attacker goal, such as stealing the model, into the paths that reach it.
A worked example
A company is launching a support agent that reads orders and issues refunds. The architect draws a data flow diagram with trust boundaries between the user, the agent, the order database and the refund API.
STRIDE is applied at each boundary, the OWASP LLM Top 10 serves as a checklist, ATLAS shapes a red team test, and CVE records are tracked for the serving software.
How the exam asks about this
- "A red team wants to map its planned attacks on an ML model to known adversary tactics and real case studies. Which of the following resources should it use?" MITRE ATLAS.
- "A risk committee needs a broad catalogue of AI risks, including societal harms and misuse, drawn from many published frameworks. Which of the following is the best resource?" The MIT AI Risk Repository.
- "A developer reviewing a new LLM chatbot wants a ranked list of the most critical application risks. Which of the following should the developer consult?" The OWASP Top 10 for LLM applications.
What to remember for the exam
- OWASP LLM Top 10 covers applications built on language models; OWASP ML Security Top 10 covers attacks on models and training.
- The MIT AI Risk Repository is a broad, sorted database of risks from many sources, using causal and domain taxonomies.
- MITRE ATLAS is the AI counterpart of ATT&CK, with tactics, techniques, mitigations and case studies.
- ATLAS adds the AI Model Access and AI Attack Staging tactics.
- The CVE AI Working Group concerns identifying and recording specific vulnerabilities in AI systems.
A top 10 list is a starting checklist, not proof that a system is secure. Combine a method such as STRIDE with the AI-specific catalogues.
Common mistakes
Choosing ATT&CK for an AI attack scenario. When the scenario involves attacks on models, training data or AI services, ATLAS is the better fit.
Treating the MIT repository as a list of exploits. It is a risk catalogue that includes social and safety risks, suited to wide risk discovery rather than attacker procedures.
Expecting a CVE for every model weakness. CVE records describe specific flaws in specific products, while bias or hallucination are managed through risk processes.
Check yourself
Which two ATLAS tactics have no direct counterpart in ATT&CK?
AI Model Access and AI Attack Staging.
What does STRIDE elevation of privilege look like in an AI agent?
A prompt injection that makes the agent use tools or data the requesting user is not allowed.
Which OWASP list would you use to review a fraud classifier's training pipeline?
The OWASP Machine Learning Security Top 10, which covers attacks such as data poisoning, model inversion and model theft.
Topic 1 of 7 in Securing AI Systems
Test yourself on Securing AI Systems
The full CompTIA SecAI+ CY0-001 question bank has 654 questions, with timed practice exams and an explanation for every answer.
This study guide is independent and is not affiliated with, endorsed by, or sponsored by CompTIA. Vendor names, certification names and exam codes are trademarks of their respective owners and are used for identification only.