Incident Detection · Topic 2 of 4

Wireshark, Performance Monitoring and Anomaly Detection

Learn Wireshark filters for spotting suspicious traffic, bandwidth and performance metrics, and behavior based anomaly detection for CND 312-38.

  • 5 min
In a nutshell
  • Capture filters (BPF) limit what is saved, display filters limit what is shown.
  • Follow TCP Stream, Conversations and Protocol Hierarchy are key Wireshark views.
  • SNMP uses UDP 161 for polling and 162 for traps, and SNMPv3 is the secure version.

What it is

Wireshark is a packet analyzer. It captures frames from a network interface or opens a saved capture, and shows each protocol layer in readable form. A defender uses it to confirm what an alert really means.

Performance and bandwidth monitoring watches how well the network is working. Anomaly detection with behavior analysis finds threats by spotting activity that departs from learned normal behavior, instead of matching known signatures.

CaptureWireshark or sensorFilternarrow to the suspectAnalyzeprotocol and payloadDecidefalse alarm or incident

How it works

Analyzing suspicious traffic with Wireshark

Wireshark has two kinds of filters. They use different syntax.

FilterWhen appliedSyntax styleExample
Capture filterBefore capture, limits what is savedBerkeley Packet Filter (BPF)host 10.0.0.5 or port 53
Display filterAfter capture, limits what is shownWireshark field namesip.addr == 10.0.0.5 or tcp.port == 80

Useful display filters:

GoalFilter
One hostip.addr == 10.0.0.5
DNS onlydns
HTTP onlyhttp
SYN packets (possible scan or flood)tcp.flags.syn == 1 && tcp.flags.ack == 0
TCP resetstcp.flags.reset == 1
ICMPicmp
Exclude a host!(ip.addr == 10.0.0.5)

Wireshark features a defender uses:

  • Follow TCP Stream rebuilds a conversation, which can show plain text content.
  • Statistics menus, such as Conversations, Endpoints and Protocol Hierarchy, show top talkers and the protocol mix to compare with the baseline.
  • Expert Information flags retransmissions, resets and malformed packets.
  • Color rules highlight problem packets.

What suspicious traces look like:

PatternLikely meaning
Many SYN packets to many ports from one source, few repliesPort scan
Many SYN packets with no ACK completionSYN flood
Many ARP replies claiming one MAC for several IPsARP spoofing
Regular small packets to one external hostBeaconing
Cleartext credentials in HTTP, FTP or TelnetWeak protocol in use
Very large or odd DNS queriesDNS tunneling

Wireshark is a tool for deep inspection of selected traffic. For constant watching across a network use sensors and flow data.

Network performance and bandwidth monitoring

MetricMeaning
BandwidthMaximum capacity of a link
ThroughputActual data rate achieved
UtilizationPercentage of capacity in use
LatencyDelay for a packet to travel
JitterVariation in delay (hurts voice and video)
Packet lossPercentage of packets that never arrive

Tools and sources: SNMP polling (UDP 161) and traps (UDP 162) from devices, flow data (NetFlow, IPFIX, sFlow) from routers, and active probes that test paths. Dashboards show trends and alert on thresholds.

Security value: a sudden jump in outbound traffic can indicate exfiltration, saturated links can indicate a DoS attack, and a high load on a host can indicate cryptomining. Quality of service (QoS) policies prioritize critical traffic, and traffic shaping can limit unwanted use.

Use SNMPv3, which adds authentication and encryption. SNMPv1 and v2c send a community string in clear text.

Network anomaly detection with behavior analysis

Signature based detection finds known bad patterns, but it misses new attacks. Anomaly detection learns a model of normal and flags deviations.

ApproachHow it works
StatisticalFlags values outside normal ranges for a metric
Machine learningLearns patterns from past data and scores new activity
User and entity behavior analytics (UEBA)Profiles each user and device and flags unusual behavior, such as a login at a rare hour
Network traffic analysis (NTA)Applies behavior models to flows and packets

Strengths and limits:

StrengthLimit
Can find unknown or zero day activityMore false positives than signatures
Finds insider and stolen credential misuseNeeds a clean training baseline and time to learn
Does not need a signatureA slow attacker can shift the baseline

Anomaly detection works best together with signatures, threat intelligence and human review.

Learnnormal behaviorScoreeach new eventAlerthigh deviationTunereduce false positives

A worked example

An analyst receives an alert for a possible scan. In Wireshark, she applies tcp.flags.syn == 1 && tcp.flags.ack == 0 and sees one internal host sending SYN packets to hundreds of ports on a server within seconds, with most answered by resets.

The pattern matches a port scan from an internal host. She checks the host owner, isolates it and escalates, since an internal scan may signal a compromised machine.

How the exam asks about this

  • "Which Wireshark filter type is applied before capture and uses BPF syntax?" → A capture filter.
  • "Which measure describes variation in packet delay?" → Jitter.
  • "Which approach flags activity that departs from learned normal behavior?" → Anomaly detection (behavior analysis, such as UEBA).

What to remember for the exam

  • Capture filters (BPF) limit what is saved, display filters limit what is shown.
  • Follow TCP Stream, Conversations and Protocol Hierarchy are key Wireshark views.
  • SNMP uses UDP 161 for polling and 162 for traps, and SNMPv3 is the secure version.
  • Latency, jitter, packet loss, throughput and utilization are the core performance metrics.
  • Anomaly detection finds new threats but produces more false positives.
Watch out

A spike in bandwidth is not always an attack. Compare against the baseline and check for a scheduled job before escalating.

Common mistakes

Mixing capture and display filter syntax. host 10.0.0.5 is a capture filter, while ip.addr == 10.0.0.5 is a display filter.

Relying only on signatures. They miss attacks that have no signature yet.

Treating anomaly alerts as proof. They show deviation, which an analyst must confirm.

Check yourself

  • Which Wireshark feature rebuilds a full conversation?

    Follow TCP Stream.

  • Which SNMP version should be used for secure monitoring?

    SNMPv3, because it provides authentication and encryption.

  • Why does anomaly detection create more false positives than signature detection?

    Because unusual but harmless activity also deviates from the learned baseline.

Topic 2 of 4 in Incident Detection

Test yourself on Incident Detection

The full EC-Council 312-38 question bank has 800 questions, with timed practice exams and an explanation for every answer.

This study guide is independent and is not affiliated with, endorsed by, or sponsored by EC-Council. Vendor names, certification names and exam codes are trademarks of their respective owners and are used for identification only.