Log Management · Topic 1 of 5
Log Management Importance and Approaches
Learn why log management matters to a SOC, the log management process, local versus centralized approaches, syslog severity, retention and integrity.
- 5 min
- Log management covers generation, collection, storage, analysis, retention and disposal.
- Syslog severities: 0 Emergency, 1 Alert, 2 Critical, 3 Error, 4 Warning, 5 Notice, 6 Informational, 7 Debug.
- Syslog uses UDP 514 by default, and TLS on 6514.
What it is
A log is a time stamped record of an event: a logon, a process start, a blocked connection, a web request. Log management is the process of generating, collecting, storing, analyzing, retaining and finally disposing of those records.
For a SOC, logs are the raw evidence. Without the right logs, kept long enough and protected from change, there is nothing to detect and nothing to investigate.
How it works
Why log management matters
| Purpose | How logs help |
|---|---|
| Threat detection | SIEM rules find attack patterns across many sources |
| Incident investigation | Logs rebuild the timeline: who, what, when, where |
| Forensics and legal evidence | Protected logs can support disciplinary or legal action |
| Compliance | Laws and standards require logging and retention, such as PCI DSS, HIPAA, SOX and GDPR |
| Accountability | Actions can be tied to a user account |
| Troubleshooting and performance | Errors and failures are visible to operations teams |
PCI DSS, for example, requires audit log history to be kept for at least 12 months, with at least the most recent three months immediately available for analysis.
Challenges
- Volume: large networks produce huge amounts of data every day.
- Variety: every vendor uses a different format and field names.
- Time: devices with unsynchronized clocks make timelines unreliable.
- Storage cost: long retention needs planning and tiering.
- Integrity: an attacker with admin rights can delete or edit local logs.
- Noise: too much low value data hides the important events.
The log management infrastructure
NIST SP 800-92 describes a log management infrastructure in three tiers:
| Tier | Role |
|---|---|
| Log generation | Hosts, devices and applications create logs |
| Log analysis and storage | Log servers or collectors receive, store and process the data |
| Log monitoring | Consoles and tools (such as a SIEM) used to review, analyze and report |
Common functions within the infrastructure:
| Function | Meaning |
|---|---|
| Parsing | Extracting fields such as time, source IP and user from a raw entry |
| Filtering | Dropping events that have no value |
| Aggregation | Merging duplicate entries into one with a count |
| Normalization | Converting fields from different formats into one standard format |
| Correlation | Linking related events from different sources |
| Rotation | Closing a log file and starting a new one at a size or time limit |
| Archival and compression | Moving older logs to cheaper storage in compact form |
| Integrity checking | Hashing logs to show they have not been altered |
Approaches to log management
| Approach | Description | Pros | Cons |
|---|---|---|---|
| Local (decentralized) | Each system keeps its own logs | Simple, no extra infrastructure | No correlation, easy for attackers to wipe, hard to search |
| Centralized | All logs forwarded to a central server or SIEM | Correlation, protection, one search point | Cost, network load, single point to secure |
| Distributed or tiered | Regional collectors forward to a central store | Scales to large or global networks | More components to manage |
| Cloud based | Logs sent to a cloud logging or SIEM service | Fast start, elastic storage | Data leaves the premises, cost grows with volume |
Most SOCs use a centralized or tiered model with a SIEM at the center.
Syslog basics
Syslog is the standard way Unix-like systems and network devices send log messages. The current standard is RFC 5424. It normally uses UDP port 514, and syslog over TLS uses port 6514.
Every message carries a facility (the source, such as kern, auth, mail or local0 to local7) and a severity:
| Level | Severity | Example |
|---|---|---|
| 0 | Emergency | System is unusable |
| 1 | Alert | Action must be taken immediately |
| 2 | Critical | Critical conditions |
| 3 | Error | Error conditions |
| 4 | Warning | Warning conditions |
| 5 | Notice | Normal but significant |
| 6 | Informational | Informational messages |
| 7 | Debug | Debug level messages |
The priority (PRI) value at the start of a message is facility × 8 + severity. For example, <34> is facility 4 (auth) with severity 2 (critical).
What to log and how to protect it
Log at least authentication, account and group changes, privilege use, process creation, policy changes, log clearing, network connections at the perimeter and access to sensitive data. Then protect the logs:
- Forward them off the host as soon as possible.
- Restrict who can read or delete them.
- Hash or sign stored logs, or use write once storage.
- Synchronize every clock with NTP.
A worked example
After a breach, investigators find that the attacker cleared the Windows Security log on the compromised server. Because the server forwarded its events to a central collector every few seconds, the logon events and the 1102 log cleared event survived there.
The central copy shows the timeline the attacker tried to erase. This is the main argument for centralized logging over local only logs.
How the exam asks about this
- "Which syslog severity level is Warning?" → 4.
- "Which log management function converts different log formats into one common format?" → Normalization.
- "What is the main weakness of keeping logs only on each local system?" → An attacker can delete them, and events cannot be correlated.
What to remember for the exam
- Log management covers generation, collection, storage, analysis, retention and disposal.
- Syslog severities: 0 Emergency, 1 Alert, 2 Critical, 3 Error, 4 Warning, 5 Notice, 6 Informational, 7 Debug.
- Syslog uses UDP 514 by default, and TLS on 6514.
- Normalization unifies formats. Aggregation merges duplicates. Correlation links related events.
- PCI DSS keeps audit logs at least 12 months, with three months immediately available.
A lower syslog number means a more severe message. Level 0 is the worst, not the least important.
Common mistakes
Logging everything at debug level. It floods storage and hides important events. Log what supports detection, investigation and compliance.
Forgetting time synchronization. Correlation across hosts fails when clocks drift.
Confusing aggregation with correlation. Aggregation combines identical events. Correlation connects different events into one story.
Check yourself
What does a syslog PRI value of 34 represent?
Facility 4 (auth) and severity 2 (critical), because 4 × 8 + 2 = 34.
Which NIST tier holds the log servers that receive and store data?
The log analysis and storage tier.
Why should logs be hashed or kept on write once storage?
To prove they have not been altered, so they remain trustworthy evidence.
Topic 1 of 5 in Log Management
Test yourself on Log Management
The full EC-Council 312-39 question bank has 700 questions, with timed practice exams and an explanation for every answer.
This study guide is independent and is not affiliated with, endorsed by, or sponsored by EC-Council. Vendor names, certification names and exam codes are trademarks of their respective owners and are used for identification only.