Remediation and Root Cause Analysis

What it is

Remediation after an incident means removing the attacker's access and fixing the weaknesses that allowed the attack. Root cause analysis (RCA) asks why the incident was possible at all, so the same failure does not repeat.

Cleaning up the visible damage is not enough. If the cause stays, the attacker, or a different one, can return through the same door.

How it works

Remediation actions

Actions depend on the incident, but typical ones are listed here.

Rebuild versus clean. Cleaning a system leaves doubt about hidden persistence. When there is any sign of deep compromise, rebuild from a known good image and restore only verified data.

Credentials need special care. Resetting a password is not enough if the attacker holds a token, a key or a forged ticket. Revoke active sessions and rotate shared secrets, and in a directory service compromise, assume that many accounts are affected.

Verifying remediation

1. Rescan or recheck the systems to confirm the weakness is closed.
2. Hunt for leftovers using the indicators from the incident, in logs and on endpoints.
3. Monitor closely for a period after recovery, with extra alerting on the affected accounts and systems.
4. Test the controls that failed, to prove they now work.
5. Confirm business function with system owners.

Releasing a system from isolation

An isolated system stays cut off until the incident lead approves release. Typical criteria and steps are:

1. Confirm it is clean: an EDR or antivirus scan and a search for the incident's indicators find nothing, or the system was rebuilt from a trusted image.
2. Confirm the cause is closed: the exploited flaw is patched and the persistence is removed.
3. Reset credentials: reset passwords, revoke sessions and rotate keys that the system used.
4. Restore access in stages: allow only management and monitoring traffic first, then the needed segments, then normal access. Re-enable disabled accounts last.
5. Monitor closely: add extra alerting for the host, and be ready to isolate it again if an indicator returns.
6. Document the approval, the time and the checks in the incident log.

Finding the root cause

A symptom is what you see, such as encrypted files. A contributing factor helps the attack succeed or spread, such as a flat network. The root cause is the underlying reason that, if fixed, would have prevented the incident, such as a missing patch process.

Common analysis methods:

An example of the five whys:

Why were files encrypted?      Ransomware ran on a file server.
Why did it run there?          An administrator account was used from a workstation.
Why was that account usable?   Its password was stolen by malware on the workstation.
Why did the malware get in?    A user opened a malicious attachment that the filter did not block.
Why did the filter miss it?    The email gateway rules were not updated and no sandbox was in use.

The last answer is a process and control gap. Fixing it prevents a family of future incidents, whereas fixing only the first answer would not.

Be careful to avoid blame. A cause such as "a user clicked a link" is a poor root cause, because it leads to blaming a person. A better cause is a missing control that should have caught the mistake.

Link to lessons learned

The findings feed back into preparation. Update the incident response plan, playbooks, detection rules, hardening standards, training and risk register. Track each corrective action with an owner and a date, and check later that it was done.

Connection to other processes

Remediation of exploited vulnerabilities joins the vulnerability management process. Changes still follow change management, though emergency changes may be fast tracked and documented afterwards.