Manage a Security Operations Environment · Topic 8 of 8
Sentinel Analytics Rules, MITRE and Anomalies
SC-200 Sentinel detections: scheduled, NRT, threat intelligence and ML rules, entity mapping, alert grouping, MITRE ATT&CK coverage and anomaly rules.
- 6 min
- Rule types: scheduled, NRT, Microsoft security, threat intelligence, ML behavioural analytics, Fusion and anomaly.
- Scheduled rules have query scheduling, lookback, threshold, entity mapping, custom details, grouping and suppression.
- NRT rules run about every minute on a single table.
What it is
Analytics rules are how Microsoft Sentinel finds threats in the data it collects. A rule runs a query or a model against your data and, when it finds a match, raises an alert. Alerts are grouped into incidents for analysts to investigate. Sentinel offers several kinds of rule, a view of how well they cover the MITRE ATT&CK framework, and anomaly rules that learn normal behaviour.
How it works
Types of analytics rule
| Rule type | How it works |
|---|---|
| Scheduled | Runs a KQL query at an interval you choose and looks back over a chosen period. The most flexible type |
| Near real time (NRT) | Runs about once a minute for very fast detection. The query uses a single table |
| Microsoft security | Creates incidents from alerts of other Microsoft products. In the Defender portal this is largely handled by Defender XDR correlation |
| Threat intelligence | Matches indicators against your events, for example the Microsoft Defender Threat Intelligence analytics rule |
| Machine learning behavioural analytics | Uses Microsoft's models to flag unusual behaviour such as anomalous sign ins |
| Fusion | Correlates alerts to detect multistage attacks |
| Anomaly | Uses models that learn normal behaviour and flag deviations |
Rule templates exist for many threats and are delivered through the Content hub. You can use a template as is, adapt it, or write a rule of your own.
Settings of a scheduled rule
A scheduled rule has several settings.
- Query, written in KQL.
- Query scheduling: how often it runs, and how far back it looks. The lookback should be at least as long as the run interval so no data is missed.
- Alert threshold: how many results trigger an alert.
- Entity mapping: links query columns to entities such as account, host, IP address, file and URL, so analysts can pivot and Sentinel can correlate.
- Custom details: adds event column values to the alert so analysts can read them without opening the logs.
- Alert details: builds a dynamic alert name and description from the results.
- Event grouping: one alert for all results, or one alert for each result.
- Alert grouping: combines alerts that share entities into one incident. For example, group alerts that involve the same account.
- Suppression: stops the rule from creating further alerts for a period after it fires.
- Incident settings: whether the alerts create incidents at all.
- Automated response: which automation rules and playbooks apply.
For any rule, test the query first, and test the rule with the results simulation where available. Tune noisy rules with watchlists that hold known good items, for example a list of approved administrator accounts, and exclude them in the query.
Managing rules
The Analytics page lists all rules with their status, type, severity and last modified time. You can turn a rule on or off, edit it, duplicate it and export it as a template. Keep rules in source control and deploy them with the Repositories feature so changes are reviewed.
MITRE ATT&CK coverage
The MITRE ATT&CK framework describes the tactics, such as initial access, persistence and exfiltration, and the techniques that adversaries use. In Sentinel you tag analytics rules, hunting queries and custom detections with tactics and techniques. The MITRE ATT&CK coverage page shows a matrix with the techniques your active rules cover and those they do not.
Use the matrix to:
- See attack vector coverage and where your detections are thin.
- Choose new rules or content hub solutions that cover the gaps.
- Compare active rules against the available templates.
SOC optimization recommendations draw on the same idea, suggesting detections for techniques you are exposed to but do not cover.
Anomalies
Sentinel includes anomaly rules that use machine learning to learn what is normal in your environment and flag deviations. Examples include unusual volumes of sign ins, a user accessing resources they do not normally use, or abnormal data transfer.
Key points:
- Anomaly rules are provided in templates and run in flighting mode at first so you can review the results without raising alerts.
- Their results are stored in the Anomalies table.
- You can customise the thresholds and parameters of an anomaly rule to fit your environment, which is how you reduce noise.
- Anomalies are not alerts by themselves. You can use them as a signal in hunting and in your own analytics rules, for example to alert when a user shows several anomalies together.
User and entity behaviour analytics (UEBA) builds profiles of users and devices, and adds context such as peer comparison and anomaly scores to investigations.
Choosing the rule type
| The need | Rule |
|---|---|
| A custom detection with a join over several tables | Scheduled rule |
| Detect a very specific event within a minute | NRT rule |
| Match events against known malicious IP addresses | Threat intelligence rule |
| Spot behaviour that is unusual for a user | Anomaly rule or UEBA |
| See which attack techniques you cannot detect | The MITRE ATT&CK coverage page |
| Stop the same alert firing again for an hour | Suppression |
| Show the account name in the alert | Entity mapping |
A worked scenario
A SOC wants to detect many failed sign ins followed by a success. A scheduled rule runs every 10 minutes with a 20 minute lookback, maps the account and IP address as entities, adds the count as a custom detail, groups alerts by account into one incident, and suppresses further alerts for an hour. A watchlist of approved penetration testing addresses is excluded in the query. The team opens the MITRE coverage page and sees few detections for exfiltration, so it installs a content hub solution that adds rules for it. It turns on anomaly rules in flighting mode, reviews them for two weeks, and tunes the thresholds.
What to remember for the exam
- Rule types: scheduled, NRT, Microsoft security, threat intelligence, ML behavioural analytics, Fusion and anomaly.
- Scheduled rules have query scheduling, lookback, threshold, entity mapping, custom details, grouping and suppression.
- NRT rules run about every minute on a single table.
- The MITRE ATT&CK coverage page shows which techniques your rules cover.
- Anomaly rules learn normal behaviour, start in flighting mode, write to the Anomalies table and can be customised.
Entity mapping is what lets analysts pivot from an alert to an account, host or IP address, and lets Sentinel group alerts.
How the exam asks about this
- "Which feature shows the attack techniques your rules do not yet detect?" The MITRE ATT&CK coverage page.
- "Which rule type detects deviation from learned normal behaviour?" An anomaly rule.
- "Stop a rule creating more alerts for a period after it fires." Suppression.
- "Show the number of failures in each alert." Custom details.
Common mistakes
Using a lookback shorter than the run interval. Events can be missed.
Skipping entity mapping. Alerts cannot be grouped or pivoted properly.
Turning on anomaly rules without review. Use flighting mode first and tune.
Writing a joined query as NRT. It supports a single table only.
Check yourself
What does entity mapping do?
It links query columns to entities such as accounts, hosts and IP addresses so alerts can be investigated and grouped.
Where do anomaly rule results go?
The Anomalies table.
What does the MITRE ATT&CK coverage page show?
The techniques covered by your analytics rules, hunting queries and detections, and the gaps.
Topic 8 of 8 in Manage a Security Operations Environment
Test yourself on Manage a Security Operations Environment
The full Microsoft SC-200 question bank has 650 questions, with timed practice exams and an explanation for every answer.
This study guide is independent and is not affiliated with, endorsed by, or sponsored by Microsoft. Vendor names, certification names and exam codes are trademarks of their respective owners and are used for identification only.