Detection · Topic 6 of 7
Network and Service Logs
Choose and set up VPC Flow Logs, Resolver query logs, load balancer, S3, WAF and CloudFront logs, and centralise them with Athena and Security Lake.
- 5 min
- Flow logs record traffic metadata with ACCEPT or REJECT, never payloads, and they skip DNS server, DHCP and metadata traffic.
- Resolver query logs show DNS lookups from your VPCs and help detect exfiltration and malware domains.
- ALB access logs and S3 server access logs go to S3 and must be switched on.
What it is
CloudTrail shows API calls, but it does not show packets, DNS lookups or web requests. Those come from other logs that you must enable and send somewhere. This page covers the main network and service logs, and how to collect them in one place with sensible retention.
The exam asks which log answers a given question, and how to store it so that it is cheap, searchable and tamper resistant.
How it works
VPC Flow Logs
VPC Flow Logs record metadata about IP traffic at a VPC, a subnet or a single network interface. They can be sent to CloudWatch Logs, S3 or Amazon Data Firehose. They do not contain packet payloads.
A default record has these fields in order: version, account, interface, source address, destination address, source port, destination port, protocol, packets, bytes, start, end, action and log status.
2 111122223333 eni-0a1b2c3d 10.0.1.15 203.0.113.50 49152 22 6 20 4000 1700000000 1700000060 REJECT OKThis line shows a TCP (protocol 6) connection to port 22 that was rejected. Custom formats add fields such as the flow direction, the original packet addresses behind a NAT device and the TCP flags.
Some traffic is never recorded, including queries to the Amazon DNS server, DHCP traffic, requests to the instance metadata address and traffic to the Amazon Time Sync address.
An ACCEPT or REJECT entry tells you the result but not which control decided it. A security group allows return traffic automatically, so an inbound ACCEPT followed by an outbound REJECT for the reply points at a network ACL.
DNS logs
| Log | Records | Destination |
|---|---|---|
| Route 53 Resolver query logs | DNS queries from resources in your VPCs, including answers | CloudWatch Logs, S3 or Firehose |
| Route 53 public DNS query logging | Queries answered for a public hosted zone | CloudWatch Logs |
Resolver query logs show which instance asked for which domain. They help you find command and control lookups and data exfiltration over DNS.
Application and service logs
| Service | Log | Notes |
|---|---|---|
| Application Load Balancer | Access logs to S3 | Off by default, bucket policy must allow the load balancer |
| Amazon S3 | Server access logs, or CloudTrail data events | Access logs are delivered on a best effort basis, data events are more complete |
| Amazon CloudFront | Standard logs and real time logs | Real time logs go through Kinesis Data Streams |
| AWS WAF | Web ACL logs | Delivered to CloudWatch Logs, S3 or Firehose, and the destination name must start with aws-waf-logs- |
| Amazon API Gateway | Access and execution logs | Configured per stage |
| AWS Network Firewall | Alert, flow and TLS logs | Sent to S3, CloudWatch Logs or Firehose |
| Amazon EKS | Control plane logs | Enable the log types you need |
WAF logs can redact chosen fields, such as authorization headers, before delivery. Use that to keep secrets out of the log store.
CloudWatch Logs protection
- A log group keeps data forever unless you set a retention period, so set one to match policy.
- Encrypt log groups with a KMS key, and allow the CloudWatch Logs service in the key policy.
- Data protection policies detect and mask sensitive data such as personal identifiers as it is ingested.
- Use subscription filters to forward logs to another account or to Firehose.
Centralising and analysing
The common design is a log archive account that owns S3 buckets with Object Lock and a lifecycle rule. Workload accounts deliver logs there directly, or through Firehose.
Amazon Security Lake automates part of this. It collects supported AWS logs and findings from an organization into S3 buckets that you own, converts them to the Open Cybersecurity Schema Framework (OCSF) and Apache Parquet format, and lets subscribers query or receive them. Use it when the question asks for a normalised, multi account security data lake with minimal custom pipeline work.
Athena queries data in S3 with SQL. Partition projection on log tables, partitioned by date, keeps queries fast and cheap.
Choosing the destination
| Need | Destination |
|---|---|
| Real time alarms on log content | CloudWatch Logs |
| Cheap long term retention and ad hoc SQL | S3 with Athena |
| Streaming to a SIEM or transformation | Firehose |
| Normalised multi account security data | Security Lake |
What to remember for the exam
- Flow logs record traffic metadata with ACCEPT or REJECT, never payloads, and they skip DNS server, DHCP and metadata traffic.
- Resolver query logs show DNS lookups from your VPCs and help detect exfiltration and malware domains.
- ALB access logs and S3 server access logs go to S3 and must be switched on.
- WAF log destinations must be named with the
aws-waf-logs-prefix. - S3 server access logs are best effort, while CloudTrail data events give more complete object level auditing.
- CloudWatch Logs retention defaults to never expire, so set it, and use data protection policies to mask sensitive data.
- Security Lake normalises logs to OCSF in your own S3 buckets across an organization.
- Use a central log archive account with Object Lock for tamper resistant retention.
For "who connected to which IP and was it allowed", choose VPC Flow Logs. For "which domain did the instance look up", choose Route 53 Resolver query logs.
Common mistakes
Expecting flow logs to show content. They show addresses, ports and counts, so use traffic mirroring or application logs for payloads.
Assuming a REJECT came from the security group. Security groups only allow, and network ACLs can deny, so check both.
Forgetting the destination permissions. Flow logs need an IAM role for CloudWatch Logs or a bucket policy for S3, otherwise nothing arrives.
Leaving log groups at never expire. Storage costs grow and sensitive data is kept longer than policy allows.
Turning on a log source does not backfill earlier activity, so enable logs before an incident, not during one.
Check yourself
Which log shows DNS queries made by instances in a VPC?
Route 53 Resolver query logs, delivered to CloudWatch Logs, S3 or Firehose.
What is Security Lake used for?
It collects and normalises security logs and findings from many accounts into your own S3 buckets in OCSF format for querying.
An inbound flow is ACCEPT and the reply is REJECT. Which control is likely responsible?
A network ACL, because security groups are stateful and allow replies automatically.
Topic 6 of 7 in Detection
Test yourself on Detection
The full AWS SCS-C03 question bank has 858 questions, with timed practice exams and an explanation for every answer.
This study guide is independent and is not affiliated with, endorsed by, or sponsored by AWS. Vendor names, certification names and exam codes are trademarks of their respective owners and are used for identification only.