AI Technologies and Controls · Topic 3 of 7
Securing the AI Life Cycle
Security across the AI life cycle: model selection, secure training and fine-tuning, independent validation, release gates, change control and retirement.
- 5 min
- Build security into every stage, with gates between stages.
- Select models on fitness, explainability, provenance, licence and security, not capability alone.
- Fine-tuning can weaken safety behaviour; re-evaluate after tuning.
What it is
The AI life cycle runs from the first idea for a use case to the day a model is retired. Security and risk controls belong at every stage, just as a secure software development life cycle builds security into code rather than testing it in at the end.
The security manager advises on the risks at each stage and makes sure gates exist where a model cannot move forward without meeting defined requirements. The manager does not train models, but does decide what evidence a model needs before it is trusted.
How it works
Stages and their security concerns
| Stage | Security and risk concerns |
|---|---|
| Define | Intended purpose, risk tier, regulatory category, acceptance thresholds, security and privacy requirements |
| Select | Fit for purpose, provenance, licence, vendor terms, known weaknesses, explainability needs |
| Prepare data | Lawful basis, quality, bias, minimization, integrity |
| Train or fine-tune | Secure, segregated environment; access control; reproducibility |
| Validate | Independent testing against thresholds: performance, robustness, safety, bias, security |
| Release | Approval gate, signed artifacts, documentation, staged rollout, rollback plan |
| Operate | Monitoring, drift detection, incident handling, user feedback |
| Change | Retraining and updates go through change control and revalidation |
| Retire | Decommission endpoints, revoke access, archive records, dispose of data and model |
Model selection
Teams choose between using a vendor model as it is, fine-tuning an existing model, or building their own. The decision weighs capability against control.
| Criterion | Question to ask |
|---|---|
| Fit for purpose | Does it meet accuracy and quality thresholds for this use? |
| Explainability | Does the use require decisions to be explained to individuals or regulators? A simpler model may be more appropriate |
| Provenance | Who built it, from what data, and is that documented in a model card? |
| Licence and terms | May it be used commercially; what happens to our data? |
| Security posture | Results of red teaming and safety evaluation; known weaknesses |
| Operating model | Hosted where, by whom, with what control? |
The most capable model is not automatically the right one. A high-risk decision about people may need a model whose reasoning can be explained and audited.
Training and fine-tuning
Training environments hold the most sensitive data and produce the most valuable artifacts. Controls include segregation from production, least-privilege access to datasets, approved and scanned base models and libraries, versioning of data, code and configuration so results can be reproduced, and recording lineage in the model registry.
Fine-tuning changes a model's behaviour, including its safety behaviour. A fine-tuned model must be re-evaluated for safety and security, not only for task performance.
Validation
Validation confirms the model meets the thresholds set at the start, using independent test data that was never used for training. For higher-risk models, validation is done or reviewed by people independent of the developers, a separation of duties familiar from model risk management in banking.
Validation covers performance, robustness, bias and fairness, safety and security testing, and confirms documentation such as the model card is accurate. Results go to the business owner, who decides with the evidence whether to accept the model.
Release and change control
A release gate checks that validation passed, documentation is complete, the risk assessment is current, monitoring is ready and a rollback plan exists. Only then is the approved version promoted from the registry.
Retraining, prompt changes, new tools, new retrieval sources and vendor model upgrades are all changes. They follow change management with revalidation proportionate to risk. A small prompt change can alter behaviour as much as retraining.
Retirement
Retirement removes endpoints, revokes credentials and tool access, archives records required for audit or regulation, and disposes of models, embeddings and data according to retention rules. The inventory and AI BOM are updated so the model does not linger in use somewhere.
A worked example
Data scientists fine-tune an open-source model to draft responses to insurance claims. They report excellent accuracy on their own test set and want to deploy next week.
The security manager points to the release gate: the test set overlaps with training data, no independent validation was done, and no safety evaluation was run after fine-tuning. The model is validated by the model risk team on held-out data, red-teamed for unsafe or misleading advice, and released to a pilot group with claims handlers reviewing every draft.
How the exam asks about this
- "Who should validate a high-risk model before deployment?" A function independent of the model developers.
- "When should acceptance thresholds for a model be defined?" At the start, when requirements are set, before training and testing.
- "A team changes the system prompt of a production assistant. What should happen?" Treat it as a change: follow change control and revalidate according to risk.
What to remember for the exam
- Build security into every stage, with gates between stages.
- Select models on fitness, explainability, provenance, licence and security, not capability alone.
- Fine-tuning can weaken safety behaviour; re-evaluate after tuning.
- Validate on independent data, by people independent of the developers for high-risk models.
- Prompt changes, retraining, new tools and vendor upgrades all go through change control.
Testing a model on data it was trained on overstates its performance. Validation data must be kept separate from training data.
Common mistakes
Choosing the most powerful model by default. Explainability, data handling and risk tier may point to a simpler or privately hosted model.
Letting developers sign off their own models. For higher-risk AI, independent validation is the control that catches optimistic results.
Treating prompt and configuration edits as minor. They can change behaviour significantly and must be tested.
Check yourself
Why must validation use independent test data?
Data used in training makes the model look better than it will perform on new data, so separate data gives an honest result.
Why re-evaluate safety after fine-tuning?
Fine-tuning can change or weaken the model's safety behaviour as well as its task performance.
What does a release gate check?
That validation passed, documentation and the risk assessment are current, monitoring is ready and a rollback plan exists.
Topic 3 of 7 in AI Technologies and Controls
Test yourself on AI Technologies and Controls
The full ISACA AAISM question bank has 747 questions, with timed practice exams and an explanation for every answer.
This study guide is independent and is not affiliated with, endorsed by, or sponsored by ISACA. Vendor names, certification names and exam codes are trademarks of their respective owners and are used for identification only.