Risk Response and Reporting · Topic 6 of 9
Control Testing Methodologies
CRISC control testing: design versus operating effectiveness, inquiry, observation, inspection, reperformance, sampling and tester independence.
- 5 min
- Test design first; a badly designed control fails without operating tests.
- Evidence strength rises from inquiry to observation to inspection to reperformance.
- Samples must cover the whole period and a complete population.
What it is
Control testing collects evidence that a control is designed properly and works as intended. Without testing, residual risk ratings rest on assumptions.
Testing answers two separate questions. A test of design asks whether the control, if it runs as described, would address the risk. A test of operating effectiveness asks whether it actually ran as described, consistently, over a period of time.
How it works
Design versus operating effectiveness
| Test | Question | Typical method |
|---|---|---|
| Design | Would this control reduce the risk if it worked? | Walkthrough of one transaction, review of procedures and settings |
| Operating effectiveness | Did the control work every time it should have, over the period? | Sample of occurrences across the period, reperformance, system evidence |
A control that fails the design test does not need operating testing; it must be redesigned first.
Testing techniques
| Technique | What it is | Strength of evidence |
|---|---|---|
| Inquiry | Asking the control owner or staff | Weakest; never enough on its own |
| Observation | Watching the control being performed | Moderate; people may act differently when watched |
| Inspection | Examining documents, records, logs, settings | Stronger, especially for system generated evidence |
| Reperformance | Tester independently performs the control and compares results | Strongest |
Tests usually combine techniques. Inquiry is used to understand the control; inspection and reperformance provide the evidence.
Sampling
Testing every occurrence is often impractical, so testers take a sample.
| Approach | Description |
|---|---|
| Statistical sampling | Random selection allows conclusions about the whole population with known confidence |
| Judgmental sampling | Tester selects items based on risk, such as high-value or unusual transactions |
| Full population testing | Data analytics test every record, possible when data is available electronically |
The sample must come from the whole period under review and from a complete population. If the population list is incomplete, the conclusion is unreliable.
Automated controls can often be tested with a single example per setting, provided general IT controls such as change management are effective, because the system applies the logic the same way every time.
Who tests
| Approach | Strength | Weakness |
|---|---|---|
| Control self-assessment (CSA) | Control owners test their own controls; builds ownership and awareness | Less independent; may be optimistic |
| Second line testing | Risk or compliance teams test with more independence | Needs resources and skills |
| Internal audit | Independent assurance to the board | Periodic, not continuous |
| Continuous controls monitoring | Automated checks run frequently on system data | Works only for controls with good data |
The independence of the tester affects how much the results can be relied on. CSA is useful but should be validated by an independent review for higher-risk controls.
Rating deficiencies
When a test finds failures, the tester rates the deficiency by its effect on the risk. In financial reporting under SOX, the levels are deficiency, significant deficiency and material weakness. More generally, the practitioner asks whether the failure is isolated or systematic, whether compensating controls exist, and how much residual risk has increased.
Results go to the control owner and risk owner, issues are logged with action plans, and risk ratings are updated.
Frequency
Higher-risk and manual controls are tested more often. Testing is also triggered by system changes, incidents and new threats. Penetration testing and vulnerability scanning are specialized forms of testing for technical controls.
A worked example
The control "user access is removed within 1 working day of a leaver's last day" is tested. The design walkthrough shows HR sends a daily leavers list to the IT service desk, which disables accounts. The design looks adequate.
For operating effectiveness, the tester takes a random sample of 25 leavers from across the year, using the HR system as the complete population. Inspection of directory logs shows that 4 accounts stayed active for more than a week, all from contractors who are not in the HR system. The deficiency is systematic for contractors, so the issue is logged, the access risk rating is raised, and the control is redesigned to include the contractor management system.
How the exam asks about this
- "Which testing technique provides the STRONGEST evidence?" Reperformance by the tester.
- "A control owner states the control works. What should the practitioner do NEXT?" Obtain evidence through inspection or reperformance; inquiry alone is not sufficient.
- "What is the MAIN limitation of control self-assessment?" Reduced independence, so results may need independent validation.
What to remember for the exam
- Test design first; a badly designed control fails without operating tests.
- Evidence strength rises from inquiry to observation to inspection to reperformance.
- Samples must cover the whole period and a complete population.
- Independence of the tester affects reliance on results.
- Test failures should update risk ratings and create tracked issues.
Inquiry alone never proves a control works. If the only evidence is that someone said so, the BEST next step is to gather objective evidence.
Common mistakes
Sampling only from recent months. Operating effectiveness covers the whole period under review.
Trusting an incomplete population. Missing items, like contractors outside HR, hide failures.
Treating CSA results as independent assurance. They show ownership but need validation for high-risk controls.
Check yourself
What is the difference between a test of design and a test of operating effectiveness?
A test of design checks whether the control would address the risk if it worked; a test of operating effectiveness checks it actually worked consistently over the period.
Why can an automated control often be tested with very few samples?
Because the system applies the same logic every time, as long as change management and other general IT controls are effective.
Which testing technique gives the weakest evidence?
Inquiry, because it relies on what people say rather than objective evidence.
Topic 6 of 9 in Risk Response and Reporting
Test yourself on Risk Response and Reporting
The full ISACA CRISC question bank has 1,000 questions, with timed practice exams and an explanation for every answer.
This study guide is independent and is not affiliated with, endorsed by, or sponsored by ISACA. Vendor names, certification names and exam codes are trademarks of their respective owners and are used for identification only.