Security, Compliance and Governance · Topic 3 of 5
AI Security Threats and Protection
Understand security and privacy for AI systems: application security, threat detection, vulnerability management, prompt injection and data poisoning.
- 4 min
- Application security covers authentication, input validation, secret handling and output checking.
- Prompt injection hides instructions in input, and the main defences are Guardrails, input checks and least privilege.
- Data poisoning corrupts training or retrieval data, and data source controls reduce the risk.
What it is
AI systems face the usual threats against cloud applications, and some extra ones that are specific to models. An attacker may target the application, the infrastructure, the data or the model itself.
A sound approach covers four areas: application security, threat detection, vulnerability management and infrastructure protection.
How it works
Application security
The application that wraps a model is often the weakest point. Protect it like any internet facing app.
- Authenticate and authorise every request.
- Validate and limit input size and type.
- Do not trust model output blindly. If it is used to build a query or a command, check it first.
- Keep secrets in a secrets service, not in prompts or code.
- Use a web application firewall, such as AWS WAF, in front of public endpoints.
AI specific threats
| Threat | What happens | Defence |
|---|---|---|
| Prompt injection | Input contains instructions that override the intended behaviour | Guardrails, input checks, separating instructions from data, least privilege |
| Jailbreaking | Prompts bypass safety rules | Guardrails, testing, monitoring |
| Data poisoning | Bad records are slipped into training or retrieval data | Control data sources, validate and version data |
| Sensitive data leakage | The model reveals private data from prompts or training | Remove or mask sensitive data, access control, output filters |
| Model theft or misuse | Someone copies or abuses the model or its API | Strong access control, rate limits, monitoring |
| Excessive agent permissions | An agent can call tools it should not | Least privilege, human approval for risky actions |
The exam stresses prompt injection and data leakage. Remember that models cannot reliably tell instructions from data, so design the application to limit the harm if they are confused.
Threat detection
Detection finds attacks and misuse as they happen.
| Service | Role |
|---|---|
| Amazon GuardDuty | Threat detection that analyses account and network activity for suspicious behaviour |
| AWS CloudTrail | Records API calls for investigation |
| Amazon CloudWatch | Logs, metrics and alarms |
| Amazon Bedrock model invocation logging | Records prompts and responses where policy allows |
Set alerts for unusual patterns, such as a sudden jump in requests or calls from unexpected locations.
Vulnerability management
Vulnerability management finds and fixes weaknesses before attackers use them. Amazon Inspector scans workloads, such as compute instances, container images and functions, for known software vulnerabilities. Keep software and dependencies patched, and scan any container images used to host models.
Infrastructure protection
Reduce exposure of the underlying systems.
| Control | Effect |
|---|---|
| Amazon VPC with private subnets | Keeps resources off the public internet |
| Security groups and network access controls | Limit which traffic can reach them |
| AWS PrivateLink and VPC endpoints | Private paths to AWS services |
| Encryption at rest and in transit | Protects data on disk and on the wire |
| Separate accounts or environments | Contain the effect of a breach |
Privacy considerations
Decide what personal data enters prompts, training sets and logs. Mask or remove it when possible, set retention limits, and control who can read logs. Check legal rules about where data can be stored and processed.
Layered defence
No single control stops every attack, so combine several.
What to remember for the exam
- Application security covers authentication, input validation, secret handling and output checking.
- Prompt injection hides instructions in input, and the main defences are Guardrails, input checks and least privilege.
- Data poisoning corrupts training or retrieval data, and data source controls reduce the risk.
- Amazon GuardDuty detects threats, and AWS CloudTrail records API activity.
- Amazon Inspector finds software vulnerabilities in workloads.
- Infrastructure protection uses VPCs, security groups, PrivateLink and encryption.
- Do not put secrets or unneeded personal data in prompts.
- Use layered, least privilege controls and human approval for risky agent actions.
Never let model output run directly as a command or database query. Treat it as untrusted input and validate it first.
Common mistakes
Assuming a system prompt prevents injection. Instructions in a prompt can be overridden, so add technical controls.
Mixing up GuardDuty and Inspector. GuardDuty detects threats from activity. Inspector scans for software vulnerabilities.
Confusing CloudTrail with CloudWatch. CloudTrail records API calls. CloudWatch gathers logs, metrics and alarms.
Ignoring the application layer. The wrapper app can leak data even if the model service is secure.
Over trusting agents. An agent with wide permissions can cause wide harm.
Check yourself
Which service detects suspicious activity in an AWS account?
Amazon GuardDuty.
Which service scans workloads for known software vulnerabilities?
Amazon Inspector.
Why should model output be validated before use?
It is untrusted and may contain injected or wrong content, so using it directly can cause harm.
Topic 3 of 5 in Security, Compliance and Governance
Test yourself on Security, Compliance and Governance
The full AWS AIF-C01 question bank has 849 questions, with timed practice exams and an explanation for every answer.
This study guide is independent and is not affiliated with, endorsed by, or sponsored by AWS. Vendor names, certification names and exam codes are trademarks of their respective owners and are used for identification only.