ISACA AAIA Practice Exam

• A. Set measurable acceptance criteria linked to the business objective Correct answer
• B. Extend the rollout to further regions to gather more performance evidence
• C. Commission an independent validation of forecast accuracy against past sales
• D. Obtain the vendor's training data documentation for the forecasting model

Option A is correct because a charter that says only 'better planning' gives nothing to test the model against, so the first step is to set measurable acceptance criteria tied to the business objective; every later assurance step depends on them. Option B is weaker because extending the rollout adds exposure without a yardstick for success. Option C is weaker because an independent validation has no pass/fail standard until criteria exist. Option D is weaker because training data documentation is useful but does not fix the missing performance targets.

• A. Judge the outputs against the acceptance criteria management defined for the use case Correct answer
• B. Enlarge the sample to 400 prompts to test whether the variation is statistically significant
• C. Ask the vendor to fix the generation settings so identical output can be re-performed
• D. Report the lack of reproducible output as a control deficiency in the drafting process

Option A is correct because generative output is non-deterministic by design, so exact re-performance is the wrong test; the auditor should judge whether the outputs meet the acceptance criteria management defined for the use case. Option B is weaker because a larger sample only confirms variation that is already expected. Option C is weaker because forcing identical output changes the tool's normal production behaviour and is not the auditor's role. Option D is weaker because variation alone is not a control deficiency.

• A. Encryption of the training dataset at rest
• B. Retention period of the raw transaction logs
• C. Completeness of the data dictionary for each field
• D. Accuracy and consistency of the fraud labels Correct answer

Option D is correct because a supervised fraud model learns directly from the labels, so inaccurate or inconsistent fraud/legitimate labels teach it the wrong patterns and undermine every output. Option A is weaker because encryption protects confidentiality, not learning quality. Option B is weaker because log retention does not affect label quality. Option C is weaker because a complete data dictionary helps understanding but does not make the labels correct.

• A. A mapping of each policy statement to the framework the organization adopted
• B. Training completion records showing that employees have been briefed on the AI policy
• C. A reconciled AI inventory showing owner, risk tier and approval status per use case Correct answer
• D. Minutes of the committee meetings held since the policy was approved

Option C is correct because a reconciled inventory showing owner, risk tier and approval status per use case is operating evidence that the policy is applied to real AI systems. Options A, B and D are weaker because a policy-to-framework mapping, training completion records and committee minutes show that governance exists or is discussed, not that it is working on actual use cases.

• A. Developers approve their own model for release Correct answer
• B. Model documentation kept in several locations
• C. No documented schedule for model retraining
• D. Hosting of the model on a third-party platform

Option A is correct because the developers approving their own model removes independent challenge and segregation of duties over a high-impact lending decision, which is the greatest concern. Options B, C and D are weaker because scattered documentation, a missing retraining schedule and third-party hosting are control weaknesses that can be managed and do not remove independent approval.

• A. Reassess the risk after any material change to the model or its data or use Correct answer
• B. Repeat the assessment on the annual enterprise risk calendar, as in prior years
• C. Replace the assessment with automated scoring of live conversations
• D. Add the chatbot to the next scheduled penetration test

Option A is correct because retraining on new data and connecting to the order system materially changed the risk, so the assessment should be refreshed after any material change to the model or its data or use. Option B is weaker because a calendar cycle can leave changes unassessed for months. Option C is weaker because automated scoring monitors behaviour but does not replace a risk assessment. Option D is weaker because a penetration test covers security only, not the full AI risk.

• A. The acceptance record does not list compensating controls
• B. The risk register has no link to the model's performance metrics
• C. The review date is set at twelve months rather than six
• D. The rating may not reflect bias among the organization's own applicants Correct answer

Option D is correct because the rating, and so the risk acceptance built on it, may not reflect bias among the organization's own applicants when the vendor tested a different country's population. Options A, B and C are weaker because missing compensating controls, a missing link to performance metrics and a twelve-month review date are secondary to the acceptance resting on unrepresentative evidence.

• A. Lawful use and quality of the training records cannot be shown Correct answer
• B. The merged set may carry conflicting retention schedules
• C. Source system owners were not told the model had been released
• D. The set is too large to reconcile record by record to source

Option A is correct because without provenance the auditor cannot show that the records were lawfully collected for this purpose or that their quality is adequate, which undermines every conclusion drawn from the model. Option B is a secondary retention issue, option C is a communication gap that does not affect the data's validity, and option D describes a testing limitation that sampling or tooling can address.

• A. Completeness checks re-performed on an extract the auditor pulls from source Correct answer
• B. A data owner sign-off confirming the report matches the dataset used
• C. The approved data-quality standard showing the 98% completeness threshold
• D. A walkthrough of the profiling script with the data scientist who wrote it

Option A is correct because re-performing the completeness checks on an extract the auditor pulls directly from source is independent, auditor-obtained evidence that does not depend on the team's own script or last month's extract. Option B only confirms a match to the dataset, not that the figure is accurate. Option C shows a threshold rather than the actual result, and option D explains how the script works without proving it produced the right output.

• A. Select a sample of deployed models and trace each back to its approval records
• B. Compare the teams' practices with a published industry methodology
• C. Obtain the approved AI life cycle standard and its stage-gate requirements Correct answer
• D. Interview the lead of each data science team about the practices followed

Option C is correct because the first step is to obtain the approved AI life cycle standard and its stage gates, which establish the criteria against which the differing team practices are to be assessed. Option A tests deployed models before the criteria are known, option B substitutes an external benchmark for the organization's own approved standard, and option D collects unverified representations about practices without a baseline.

• A. Repeat the risk and impact assessment whenever a model's intended use changes Correct answer
• B. Make a data protection review mandatory at the proof-of-concept stage
• C. Commission independent validation of the model before further limit decisions
• D. Have the business owner sign off at every future stage gate of the model

Option A is correct because the failure was that every gate was passed yet the original risk and impact assessment, made for an anonymized proof of concept, was never refreshed when the use changed; requiring reassessment on any change of intended use closes that gap. Option B addresses only privacy at one stage, option C is a one-off check of this model rather than a preventive control, and option D adds sign-off that would not have caught the change in use.

• A. The training job log showing the run completed without errors
• B. A release note the data science team posted to the intranet after go-live
• C. An approved change ticket tied to the new model version's test results Correct answer
• D. The retraining schedule approved in the annual AI plan and budget

Option C is correct because an approved change ticket linked to the new version's test results shows authorization and testing took place before the model replaced the production version. Option A shows only that the job ran, option B is a notice posted after go-live and so cannot evidence prior approval, and option D shows a planned schedule, not approval of this specific change.

• A. Replace the hosted service with an in-house model the retailer can version
• B. Obtain a SOC 2 Type II report from the provider covering its model releases
• C. Set alert-rate thresholds after each update that trigger a rollback review
• D. Negotiate advance notice of model changes and test each before it goes live Correct answer

Option D is correct because the root problem is that the provider can change the model without notice and the retailer cannot test it first; negotiating advance notice and pre-release testing restores change control at its source. Option A is a costly strategic move beyond what the finding requires, option B gives general control assurance but not control over model changes, and option C is only a detective measure that acts after the live service has already changed.

• A. Review the adjusters' training records on the claims model
• B. Analyze adjuster time per case and the frequency of overrides Correct answer
• C. Confirm the procedure assigns an accountable human to each claim
• D. Compare the model's approval rate with the prior year's manual rate

Option B is correct because a 99.8% acceptance rate with decisions in seconds suggests rubber-stamping, and time per case and override frequency are direct evidence of whether oversight is genuine. Option A and option C review whether training and accountability exist on paper, not whether review is effective, and option D compares approval rates, which does not show how adjusters actually exercise judgement.

• A. Add one line item for artificial intelligence to the plan and assign it a fixed number of audit days
• B. Ask the head of data science to nominate the models that deserve audit attention
• C. Draft an audit program for the best-known customer-facing model so work can begin early in the year
• D. Survey business units and procurement records to find models in use that IT has not catalogued Correct answer

Option D is correct because the first step in planning is to establish the population: surveying business units and procurement records finds models embedded in purchased software that IT has not catalogued, so the audit universe can then be risk-ranked. Option A allocates days to an undefined, undifferentiated entry. Option B relies on one function that may not know about models bought by other units. Option C starts detailed work before the universe is complete and may focus on the wrong model.

• A. Rely on the report for the areas it covers and plan outcome testing using the bank's own decision data Correct answer
• B. Plan to re-perform the report's training-data checks on extracts the vendor would be asked to provide
• C. Lower the model's inherent risk rating in the plan because an independent party has already validated it
• D. Exclude the model from the engagement and record the missing audit right as a scope limitation

Option A is correct because the auditor can use the independent report for the areas it covers while closing the gap it leaves, namely performance for the bank's own customers, with outcome testing on the bank's own decision data, which needs no vendor audit right. Option B re-performs work that depends on vendor cooperation the contract does not provide. Option C lowers risk on the basis of a report that does not cover the bank's population. Option D abandons the engagement although alternative evidence exists.

• A. The number of vendors that supplied components to the system
• B. The consequence of its decisions for customers and how far humans review them Correct answer
• C. The sophistication of the algorithm used to build the model
• D. The total development spending approved for the system

Option B is correct because audit priority follows risk, and the impact of a system's decisions on customers, together with how much human review sits over them, determines how much harm an error could do. Options A, C and D describe supplier count, technical sophistication and spending, none of which indicates the consequence or likelihood of harm.

• A. Select decisions across the whole quarter so each of the three versions is represented in the sample Correct answer
• B. Select decisions made by the latest version, since the earlier versions are no longer serving customers
• C. Test the code review records for each version instead of drawing a sample of decisions
• D. Select the decisions that customers appealed, because they show where the model departs from policy

Option A is correct because the conclusion covers the whole quarter, so the sample must represent every version in production; otherwise decisions made by earlier versions are never tested. Option B leaves out part of the population. Option C tests development controls, not whether decisions followed credit policy. Option D is a biased sample of appealed cases that cannot support a conclusion about the whole population.

• A. Accept the report once its figures agree with the performance stated in the model documentation
• B. Re-perform the full validation using the developers' own scripts to confirm that the figures hold
• C. Weigh the validators' independence, then corroborate key results with data the auditor obtains Correct answer
• D. Treat the report as a management representation and seek written confirmation from the chief risk officer

Option C is correct because evidence from the developers is not independent, so the auditor should assess the validators' independence and objectivity and corroborate the key results with data the auditor obtains. Option A accepts the report because it agrees with documentation written by the same team. Option B re-performs the validation using the developers' own scripts, which brings the same bias and is disproportionate. Option D replaces evidence with a representation, which is weaker.

• A. The approved change ticket for the release deployed closest to the decision date
• B. The registry's current entry, which names the production version and records its validation approval
• C. The version stamp on the decision record, matched to the validation approval held for that version Correct answer
• D. A written statement from the model owner of which version was serving on that date

Option C is correct because the version stamp on the decision record ties the actual decision to the exact model version that scored it, and matching it to that version's validation approval gives direct, system-generated evidence. Option A shows only the nearest release, not the version that scored this case. Option B shows the current state, which may differ from four months ago. Option D is an unsupported assertion by an interested party.