Network Anomaly Detection

1
Network Anomaly Detection

• HIT Laboratory, SungKyunKwan Univ.
• 2006.06.17 SAT
• Jung-Yoon Kim(steal83_at_hit.skku.edu)
• Chan-Kyu Han(hedwig_at_hit.skku.edu)

2
Contents

• Project Introduction
• Background knowledge
• Scenario Proposal
• Program Architecture
• Simulation

3
Anomaly Detection

• Is the network experiencing unusual conditions?
• Call these conditions anomalies
• Anomalies can often indicate network problems
• DDoS, worms, flash crowds, outages,
• Need rapid detection and diagnosis
• The need for techniques to detect network
  anomalies in real time is increasing
• Intrusion Detection System (IDS)

4
Background knowledge

• What we want
• Volume anomalies
• Significant changes in an Origin-Destination flow
• What we have
• Link traffic measurements
• SNMP provides link measurement (bytes and
  packets)
• It is difficult to measure traffic matrix directly

5
The Subspace Method

• An approach to separate normal from anomalous
  traffic
• Normal Subspace, space spanned by the
  first k principal components
• Anomalous Subspace, space spanned by the
  remaining principal components

Residual trafficvector
Traffic vector of all links at a particular
point in time
Normal trafficvector
6
A Illustration
In general, anomalous traffic results in a large
value of
Traffic on Link 2
Traffic on Link 1
7
Detection

• Capture size of vector using squared prediction
  error
• Assuming Gaussian data, we can find boundswhich
  SPE should only exceed 1- of the time
• Result due to Jackson and Mudholkar, 1979

Traffic on Link 2
Traffic on Link 1
8
Inference Algorithm

• Tomography
• Infer volume anomalies from link traffic
  measurements
• Early Inverse
• Drawback errors in step 1 may contaminate step 2
• Inversion
• Infer OD flows X by solving btAxt
• Ill-posed linear inverse problem
• Anomaly extraction
• Extract volume anomalies X from inferred X
• Potentially computationally expensive anomaly
  detection step to high dimensional data
• Late Inverse
• Idea defer lossy inference to the last step
• Anomaly extraction
• Extract link traffic anomalies B from B
• Inversion
• Infer volume anomalies X by solving btAxt

9
Development

• Fedora Core 4
• Perl language
• MathMatrixReal module
• Duty Assignment
• Generate Anomaly Traffic source
• Backbone data
• Principle Component Analysis
• Identify Candidate Anomaly

10
Program
11
Result ?
12
Result ?
13
Constraint

• Traffic Data Matrix
• Accurate OD-flow
• Accurate of packet byte in any network?
• Detection rate

14
References

• Diagnosing Network-Wide Traffic Anomalies
• A.Lakhina, M.Crovella and C.Diot. In ACM
  SIGCOMM04
• PCA, Subspace Method
• Math-MatrixReal gt MathMatrixReal
• http//search.cpan.org/dist/Math-MatrixReal/Matrix
  Real.pmOVERLOADED_OPERATORS