Monitoring, Diagnosing, and Securing the Internet

1
Monitoring, Diagnosing, and Securing the Internet

• Yan Chen
• Department of Electrical Engineering and Computer
  Science
• Northwestern University
• Lab for Internet Security Technology (LIST)
• http//list.cs.northwestern.edu

2
(No Transcript)
3
The Spread of Sapphire/Slammer Worms
4
Current Intrusion Detection Systems (IDS)

• Mostly host-based and not scalable to high-speed
  networks
• Slammer worm infected 75,000 machines in lt10 mins
• Host-based schemes inefficient and user dependent
• Have to install IDS on all user machines !
• Mostly simple signature-based
• Inaccurate, e.g., with polymorphism
• Cannot recognize unknown anomalies/intrusions

5
Current Intrusion Detection Systems (II)

• Cannot provide quality info for forensics or
  situational-aware analysis
• Hard to differentiate malicious events with
  unintentional anomalies
• Anomalies can be caused by network element
  faults, e.g., router misconfiguration, link
  failures, etc., or application (such as P2P)
  misconfiguration
• Cannot tell the situational-aware info attack
  scope/target/strategy, attacker (botnet) size,
  etc.

6
Network-based Intrusion Detection, Prevention,
and Forensics System

• Online traffic monitoring and recording
• SIGCOMM IMC 2004, INFOCOM 2006, ToN 2007
  INFOCOM 2008
• Reversible sketch for data streaming computation
• Record millions of flows (GB traffic) in a few
  hundred KB
• Small of memory access per packet
• Scalable to large key space size (232 or 264)
• Online sketch-based flow-level anomaly detection
• IEEE ICDCS 2006 IEEE CGA, Security
  Visualization 2006
• Detect TCP SYN flooding, horizontal and vertical
  scans even when mixed
• Online stealthy botnet scan detection
• IEEE IWQoS 2007

7
Network-based Intrusion Detection, Prevention,
and Forensics System (II)

• Accurate network and distributed system diagnosis
• Overlay network monitoring and diagnosis SIGCOMM
  IMC 2003, SIGCOMM 2004, ToN 2007 SIGCOMM 2006
• End-user network diagnosis INFOCOM 2007 (2)
• Internet-scale Virtual Private Network (VPN) and
  backbone monitoring and diagnosis Work under
  submission
• Internet-scale Data Center and dist system
  profiling and diagnosis Work in progress

8
Network-based Intrusion Detection, Prevention,
and Forensics System (III)

• Large-scale botnet and P2P misconfiguration event
  situational-aware forensics
• Botnet attack target/strategy inference
  ASIACCS09
• Root cause analysis of the P2P misconfiguration/po
  isoning traffic work under submission

9
Network-based Intrusion Detection, Prevention,
and Forensics System (IV)

• Polymorphic worm signature generation detection
  IEEE Symposium on Security and Privacy 2006
  IEEE ICNP 2007

Signature 10.01
Traffic Filtering
Internet
X
X
9
10
Network-based Intrusion Detection, Prevention,
and Forensics System (V)

• NetShield vulnerability signature based NIDS for
  high performance network defense work in
  progress
• Vulnerability analysis of wireless network
  protocols and its defense work in progress

10
11
System Deployment

• Attached to a router/switch as a black box
• Edge network detection particularly powerful

Monitor each port separately
Monitor aggregated traffic from all ports
Original configuration
12
NetShield Matching with a Large Vulnerability
Signature Ruleset for High Performance Network
Defense
13
Limitations of Exploit Based Signature
Signature 10.01
Traffic Filtering
Internet
Our network
X
X
Polymorphism!
Polymorphic worm might not have exact exploit
based signature
14
Vulnerability Signature
Vulnerability signature traffic filtering
Internet
X
X
Our network
X
X
Vulnerability

• Work for polymorphic worms
• Work for all the worms which target the
• same vulnerability

15
Example of Vulnerability Signatures

• At least 75 vulnerabilities are due to buffer
  overflow
• Sample vulnerability signature
• Field length corresponding to vulnerable buffer gt
  certain threshold
• Intrinsic to buffer overflow vulnerability and
  hard to evade

Overflow!
Protocol message
Vulnerable buffer
16
Vision of NetShield
17
Motivation

• Desired Features for Signature-based NIDS/NIPS
• Accuracy (especially for IPS)
• Speed
• Coverage Large ruleset

Cannot capture vulnerability condition well!
Shield sigcomm04
Regular Expression Vulnerability
Accuracy Relative Poor Much Better
Speed Good ??
Memory OK ??
Coverage Good ??
18
Research Challenges

• Background
• Use protocol semantics to express vulnerability
• Protocol state machine predicates for each
  state
• Example ver1 methodput len(buf)gt300
• Challenges
• Matching thousands of vulnerability signatures
  simultaneously
• Sequential matching ? algorithmic parallel
  matching
• High speed parsing
• Applicability for large NIDS/NIPS rulesets

19
Outline

• Motivation
• Feasibility Study a Measurement Approach
• Given a large NIDS/NIPS ruleset, what percentage
  of the rules can be improved with protocol
  semantic vulnerability signatures?
• High Speed Parsing
• High Speed Matching for Large Rulesets.
• Evaluation
• Conclusions

20
Measure Snort Rules

• Semi-manually classify the rules.
• Group by CVE-ID
• Manually look at each vulnerability
• Results
• 86.7 of rules can be improved by protocol
  semantic vulnerability signatures.
• Most of remaining rules (9.9) are web DHTML and
  scripts related which are not suitable for
  signature based approach.
• On average 4.5 Snort rules are reduced to one
  vulnerability signature.
• For binary protocol the reduction ratio is much
  higher than that of text based ones.
• For netbios.rules the ratio is 67.6.

21
Outline

• Motivation
• Feasibility Study a Measurement Approach
• High Speed Parsing
• High Speed Matching for Large Rulesets.
• Evaluation
• Conclusions

22
Observations

• PDU ? parse tree
• Leaf nodes are integers or strings
• Vulnerability signatures mostly based on leaf
  nodes

• Observation 1 Only need to parse the fields
  related to signatures.
• Observation 2 Traditional recursive descent
  parsers which need one function call per node are
  too expensive.

23
Efficient Parsing with State Machines

• Pre-construct parsing state machines based on
  parsing trees and vulnerability signatures.
• Studied eight protocols HTTP, FTP, SMTP, eMule,
  BitTorrent, WINRPC, SNMP and DNS as well as their
  vulnerability signatures.
• Common relationship among leaf nodes.

24
Example for WINRPC

• Rectangles are states
• Parsing variables R0 .. R4
• 0.61 instruction/byte for BIND PDU

25
Outline

• Motivation
• Feasibility Study a Measurement Approach
• High Speed Parsing
• High Speed Matching for Large Rulesets.
• Evaluation
• Conclusions

26
A Matching Problem Example

• Data representations
• For all the vulnerability signatures we studied,
  we only need integers and strings
• Integer operators , gt, lt
• String operators , match_re(.,.), len(.).
• Example signature for Blaster worm

27
Matching Problem Formulation

• Suppose we have n signatures, each is defined on
  k matching dimensions (matchers)
• A matcher is a two-tuple (field, operation) or a
  four-tuple for the associate array elements.
• Efficiently report all the matched rules.
• Challenges for Single PDU matching problem (SPM)
• Large number of signatures n
• Large number of matchers k
• Large number of dont cares
• Cannot reorder matchers arbitrarily -- buffering
  constraint
• Field dependency
• Arrays, associate arrays
• Mutually exclusive fields.

28
Matching Algorithms

• Two steps
• Pre-computation decides the rule order and
  matcher order
• Divide-and-conquer comparison w/ matchers and
  combine the results efficiently
• Under each matcher m, parallel matching of all
  the rules that involve m
• Iteratively filter/combine the candidates from
  each matching.

29
Step 1 Pre-Computation

• Put the selective matchers earlier
• Observe buffering constraint field arrival
  order

30
Step 2 Iterative Matching
31
Refinement and Extension

• SPM improvement
• Allow negative conditions
• Handle array case
• Handle associate array case
• Handle mutual exclusive case
• Report the matched rules as early as possible
• Extend to Multiple PDU Matching (MPM)
• Allow checkpoints.

32
Outline

• Motivation
• Feasibility Study a Measurement Approach
• Problem Statement
• High Speed Parsing
• High Speed Matching for Large Rulesets.
• Evaluation
• Conclusions

33
Evaluation Methodology

• Fully implemented and deployed to sniff a campus
  router hosting university Web servers and several
  labs.
• Run on a P4 3.8Ghz single core PC w/ 4GB memory.
• Much smaller memory usage. E.g., http 791
  vulnerability sigs from 941 Snort rules
• DFA 5.29 GB vs. NetShield 1.08MB

34
Stress Test Results

• Traces from Tsinghua Univ. (TH) and Northwestern
  (NU)
• After TCP reassembly and preload the PDU in
  memory
• For DNS we only evaluate parsing.
• For WINRPC we have 45 vulnerability signatures
  which covers 3,519 Snort rules
• For HTTP we have 799 vulnerability signatures
  which covers 973 Snort rules.

35
Stress Test Results ParsingMatching
35
36
Conclusions

• A novel network-based vulnerability signature
  matching engine
• Through measurement study on Snort ruleset, prove
  the vulnerability signature can improve most of
  the signatures in NIDS/IPS.
• Proposed parsing state machine for fast parsing
• Propose a candidate selection algorithm for
  matching a large number of vulnerability
  signature simultaneously

37
With Our Solutions
Regular Expression Vulnerability
Accuracy Relative Poor Much Better
Speed Good Even faster
Memory OK Better
Coverage Good Similar
Build a better Snort alternative
38
Backup
39
Architecture of Network IDS
Signature matching ( protocol parsing when
needed)
Protocol identification
TCP reassembly
Packet capture libpcap
Packet stream
40
Observations

• Observation 1 Most matchers are good.
• After matching against them, only a small number
  of signatures can pass (candidates).
• String matchers are all good, and most integer
  matchers are good.
• We can buffer bad matchers to change the matching
  order.
• Observation 2 NIDS/NIPS will report all the
  matched rules regardless the ordering. Different
  from firewall rules.

40
41
Observation

• PDU ? parse tree
• Leaf nodes are integers or strings
• Vulnerability signature mostly based on leaf nodes

Only need to parse the fields related to
signatures

• Traditional recursive descent parsers (BINPAC)
  which need one function call per node are too
  expensive.

42
Limitations of Regular Expression Signatures
Signature 10.01
Traffic Filtering
Internet
Our network
X
X
Polymorphism!
Polymorphic attack (worm/botnet) might not have
exact regular expression based signature
43
Reason
Shield
RE
X
Cannot express exact condition
Can express exact condition

• Regular expression is not power enough
• to capture the exact vulnerability condition!

44
Outline

• Motivation
• Feasibility Study a measurement approach
• Problem Statement
• High Speed Parsing
• High Speed Matching for massive vulnerability
  Signatures.
• Evaluation
• Conclusions

45
What Do We Do?

• Build a NIDS/NIPS with much better accuracy and
  similar speed comparing with Regular Expression
  based approaches
• Feasibility in Snort ruleset (6,735 signatures)
  86.7 can be improved by vulnerability
  signatures.
• High speed Parsing 2.712 Gbps
• High speed Matching
• Efficient Algorithm for matching a large number
  of vulnerability rules
• HTTP, 791 vulnerability signatures at 1Gbps

46
Network based IDS/IPS

• Accuracy (especially for IPS)
• False positive
• False negative
• Speed
• Coverage Large ruleset

Regular Expression Vulnerability
Accuracy Poor Much Better
Speed Good Good
Coverage Good Good
Regular expression is not power enough to capture
the exact vulnerability condition!
47
Stress Test Results

• Traces from Tsinghua Univ. (TH) and Northwestern
  (NU)
• After TCP reassembly and preload the PDU in
  memory
• For DNS we only evaluate parsing.
• For WINRPC we have 45 vulnerability signatures
  which covers 3,519 Snort rules
• For HTTP we have 799 vulnerability signatures
  which covers 973 Snort rules.

47