Detection · Topic 1 of 7

CloudWatch Alarms and Metric Filters

Turn log events into security alarms with CloudWatch metric filters, metric alarms and composite alarms, and know how missing data and SNS delivery behave.

5 min

In a nutshell

A metric filter turns log events into a metric, and an alarm acts on that metric. The filter does not look at logs written before it existed.
Send CloudTrail to CloudWatch Logs to alarm on API calls in near real time. CloudTrail alone does not raise alarms.
Alarm states are OK, ALARM and INSUFFICIENT_DATA, and the missing data setting decides what a silent metric means.

What it is

Amazon CloudWatch is the AWS service that stores metrics and logs and can raise alarms on them. For security work, the classic pattern is to send AWS CloudTrail events to a CloudWatch Logs log group, extract a signal with a metric filter, and alarm on the resulting metric.

This gives near real time alerting on things such as root account use, console sign in without multi factor authentication (MFA), or changes to security groups. The CIS (Center for Internet Security) AWS Foundations Benchmark popularised this list of alarms.

How it works

A metric filter watches a log group and increments a custom metric each time a log event matches the filter pattern. A metric filter only counts events that arrive after it was created, so it never back fills old log data.

An alarm watches one metric (or a metric math expression) and moves between three states: OK, ALARM and INSUFFICIENT_DATA. When the state changes, the alarm can run actions such as publishing to an Amazon SNS (Simple Notification Service) topic.

Example: alert on root account use

# AWS CLI: create the metric filter on the CloudTrail log group
aws logs put-metric-filter \
  --log-group-name CloudTrail/DefaultLogGroup \
  --filter-name RootAccountUsage \
  --filter-pattern '{ $.userIdentity.type = "Root" && $.userIdentity.invokedBy NOT EXISTS && $.eventType != "AwsServiceEvent" }' \
  --metric-transformations metricName=RootUsageCount,metricNamespace=SecurityMetrics,metricValue=1

# AWS CLI: alarm when the metric is 1 or more in five minutes
aws cloudwatch put-metric-alarm \
  --alarm-name RootAccountUsage \
  --namespace SecurityMetrics --metric-name RootUsageCount \
  --statistic Sum --period 300 --evaluation-periods 1 \
  --threshold 1 --comparison-operator GreaterThanOrEqualToThreshold \
  --treat-missing-data notBreaching \
  --alarm-actions arn:aws:sns:us-east-1:111122223333:security-alerts

The pattern uses the JSON filter syntax, where $.field refers to a field of the CloudTrail record. The --treat-missing-data notBreaching setting matters, because a security metric is empty most of the time.

Alarm building blocks

Feature	What it does	Choose it when
Metric alarm	Compares one metric to a threshold	A single signal is enough
Composite alarm	Combines other alarms with AND, OR and NOT rules	You want fewer, higher quality notifications
Anomaly detection alarm	Compares a metric to a learned band instead of a fixed number	Normal levels change by hour or day
Metric math alarm	Alarms on an expression over several metrics	You need a ratio or a sum

Missing data handling

An alarm has to decide what to do when no data points arrive in a period. There are four settings: notBreaching, breaching, ignore and missing. The default is missing, which can leave an alarm in INSUFFICIENT_DATA state, so choose notBreaching for event counters and breaching for a heartbeat metric that should never go quiet.

Other uses of the same logs

A subscription filter streams matching log events in real time to a destination such as Amazon Kinesis Data Streams, Amazon Data Firehose or an AWS Lambda function. Use it to feed a SIEM (security information and event management) tool or to run code on a pattern. A metric filter gives you counts and alarms, while a subscription filter gives you the events themselves.

CloudWatch Logs Insights lets you query log groups interactively with a purpose built query language. It is for investigation after an alert, not for alerting.

Cross account and organization use

Alarms live in the account and Region where the metric is. Security teams often need one view over many accounts, which CloudWatch cross account observability supports through a monitoring account that links to source accounts. For alerting, the usual design still sends the signal to a central place, such as an EventBridge bus or a central log account, which the next pages cover.

What to remember for the exam

A metric filter turns log events into a metric, and an alarm acts on that metric. The filter does not look at logs written before it existed.
Send CloudTrail to CloudWatch Logs to alarm on API calls in near real time. CloudTrail alone does not raise alarms.
Alarm states are OK, ALARM and INSUFFICIENT_DATA, and the missing data setting decides what a silent metric means.
A composite alarm reduces alert noise by combining several alarms with Boolean logic.
Use a subscription filter to stream logs onward, and a metric filter to count and alarm.
An SNS topic encrypted with a customer managed KMS key needs a key policy that lets CloudWatch publish. The AWS managed key aws/sns cannot be edited to allow this.
Alarm actions can notify through SNS, run EC2 or Auto Scaling actions, and open Systems Manager items.
Alarms are Regional, so a multi Region design needs the same filter and alarm in every Region you care about.

Exam tip

When a question says "notify within minutes when a specific API call occurs" and lists CloudWatch Logs metric filters next to Amazon EventBridge, both can work. Pick EventBridge if the event comes from an AWS service directly, and the metric filter if the log group is already the data source.

Common mistakes

Expecting a metric filter to search history. It only processes events delivered after creation, so use Logs Insights or Amazon Athena for older data.

Leaving missing data on the default. An event counter with no events stays in INSUFFICIENT_DATA or flaps, so set notBreaching for counters.

Forgetting the notification path. An alarm in ALARM state with no action, or an SNS topic with no confirmed subscription, tells nobody anything.

Using one Region. A trail that logs only one Region, or an alarm created in one Region, misses activity elsewhere.

Watch out

If the SNS topic uses a KMS key and the key policy does not allow CloudWatch, the alarm changes state but the email never arrives. The failed action is recorded in the alarm history.

Check yourself

Which two CloudWatch features turn a CloudTrail record into an email?
A metric filter creates a metric from matching log events, and an alarm on that metric publishes to an SNS topic with an email subscription.
Why use a subscription filter instead of a metric filter?
A subscription filter streams the matching events to Kinesis, Firehose or Lambda, while a metric filter only produces a count.
Which missing data setting suits a metric that is empty unless something bad happens?
notBreaching, so empty periods count as healthy and the alarm stays in OK.

Topic 1 of 7 in Detection

Test yourself on Detection

The full AWS SCS-C03 question bank has 858 questions, with timed practice exams and an explanation for every answer.

Subscribe Try Sample Exam

This study guide is independent and is not affiliated with, endorsed by, or sponsored by AWS. Vendor names, certification names and exam codes are trademarks of their respective owners and are used for identification only.