7' Monitoring

1
7. Monitoring Analysis Tools
2
7. Monitoring Analysis Tools

• Tremendous variety of tools are available for
  monitoring many aspects of networks
• Simple commands usually included in operating
  systems
• Free (open-source) applications
• Commercial packages and systems
• Category
• Passive monitoring tools
• Traffic Flow Analysis
• NetFlow (C), cflowd (F), FlowScan (F), Sniffer
  Pro (C), argus (F), i-Flow (C)
• Network Utilization
• MRTG (F), RMON (C)
• Visualization
• RRD (F)
• Active monitoring tools
• Network Performance
• ping (S), traceroute (S), Network Vantage (C),
  NetPerf (F), etc.

3
7. Monitoring Analysis Tools - NetFlow

• Cisco IOS NetFlow Infrastructure

Network Planning
RMON Probe
RMON Application
Accounting/Billing

• NetFlowFlowCollector
• Data Collection
• Data Filtering
• Data Aggregation
• Data Storage

• NetFlow Data Export
• Data Switching
• Data Aggregation
• Data Export

• Network Data Analyzer
• Data Presentation
• Flow Control and Configuration
• Partner Applications

Image From NetFlow PPT by Michael Lin, Cisco
Systems
4
7. Monitoring Analysis Tools - NetFlow

• NetFlow FlowCollector
• provides fast and scalable data collection from
  multiple NetFlow Export-enabled devices
• performs data volume reduction through selective
  filtering and aggregation
• stores flow information in flat files on disk for
  post-processing by consumer applications.

NetFlow Enabled Devices
NetFlow FlowCollector
NetFlow Consumer Applications
Image From NetFlow PPT by Michael Lin, Cisco
Systems
5
7. Monitoring Analysis Tools - NetFlow

• Network Data Analyzer
• Receives flow data from NetFlow FlowCollector(s)
• Performs time-based analysis and data sorting
• Configures FlowExports and FlowCollectors
• Produces histograms, bar charts, and pie charts

NetFlow FlowCollectors
NetFlow FlowAnalyzer
Image From NetFlow PPT by Michael Lin, Cisco
Systems
6
7. Monitoring Analysis Tools - cflowd

• Freely available NetFlow analysis tool from CAIDA
• Functionality
• Input
• NetFlow export data from Cisco routers
• Collect
• collect Information of flow obtained from
  NetFlow
• store uses arts file format (binary file
  format specification for storing network data)
• Analyze
• predetermined statistics text format, using ARTS
  utility (e.g., xartsprotos)
• query and visualize using java front-end

7
7. Monitoring Analysis Tools cflowd
Source http//www.caida..org
8
7. Monitoring Analysis Tools FlowScan

• Traffic Reporting Visualization Tool
• developed by Dave Plonka(U. Wisconsin)
• analyzes and reports on flow data exported by
  routers
• produces graph images which provide a continuous,
  near real-time view of the network traffic across
  a network's border
• freely available
• FlowScan binds together
• (1) a flow collection engine (a patched version
  of cflowd)
• (2) a high performance database (Round Robin
  Database - RRD)
• (3) a visualization tool (RRDtool)

9
7. Monitoring Analysis Tools FlowScan
loads and executes report modules of the
administrators choosing
Source FlowScan, Dave Plonka
10
7. Monitoring Analysis Tools arts

• ART is a binary file format specification for
  storing network data
• ART was initially developed at ANS (American
  Nuclear Society) by David Bolen (1992)
• ARTS was licensed to CAIDA (1998)
• ARTS data objects are generally composed of three
  parts
• a header, a list of attributes and a data
  section
• CAIDA has developed a C class library for ARTS
  called arts
• arts Functionality
• efficient data archival
• aggregation in the time domain (AS, net, port,
  protocol, interface..)
• version-specific formats
• support for iostreams and UNIX file descriptors

11
7. Monitoring Analysis Tools ARGUS

• Audit Record Generation and Utilization System
• A powerful flow-based, passive monitoring tool
  for IP networks
• Provides tools for various analysis of network
  activity
• Probe system argus
• Collector/Analysis tools ra, racount, ragator,
  ramon, rasort, raxml
• Developed originally by CMU in 1993, now
  coordinated by QoSient LLC as open source project
• Current Release Version 2.0.5
• Current Developing Version 2.0.6
• http//www.qosient.com/argus
• Fixed model Real-Time Flow Monitor after IETF RTFM

12
Argus Architecture
Source QoSient LLC
13
Argus Data Model

• Argus flow modeled after IPPM Framework
• Type-P and Type-P1-P2 flows
• Bidirectional flow model lt- RTFM
• Packets of Type-P
• Defined in RFC 2330 from IETF IPPM WG
• To remove the ambiguity in the definition of
  Network Performance Metrics
• The generic notion where in some contexts P will
  be explicitly defined(Type-P), partially
  defined(Type-P1-P2), or left generic
• Example
• IP-connectivity
• IP-Type-P-Connectivity, IP-Port-HTTP-Connectivity

14
Argus Flows

• An Argus Flow is simply a set of datagrams that
  share a common set of datagram attributes.
• Destination Address
• Network Addresses
• Addresses, Protocol, NSAPs, TTL, Session IDs,
  Application data, etc.
• Supports 13 simultaneous flow models, enabling
  Layer 2, 3, 4, and 5 based flow tracking and
  reporting

15
Argus Flow Models

• Layer 5
• RTP and RTCP (Type-P)
• 8-tuple  SrcIPAddr,DstIPAddr,L4Protocol,SrcPort,D
  stPort,  rh_ver, rh_seq, rh_ssrc
• Layer 4
• TCP and UDP (Type-P)
• 5-tuple  SrcIPAddr, DstIPAddr, L4Protocol,
  SrcPort, DstPort
• ESP (Type-P)
• 4-tuple  SrcIPAddr, DstIPAddr, L4Protocol, SPI
• ICMP ECHO (Type-P1-P2)
• 7-tuple  SrcIPAddr, DstIPAddr, L4P, type,
  code,id, seq             where the type is
  either ECHO REQUEST or REPLY.
• ICMP INFO TYPE (Type-P1-P2)
• 5-tuple  SrcIPAddr, DstIPAddr, L4P, type, code
              where the type is either REQUEST or
  REPLY.
• ICMP UNREACHABLE/REDIRECT (Type-P1-P2)
• Mapped to any supported Argus flow type.
• 6-tuple  SrcIPAddr, DstIPAddr, L4P, type, code,
  object     
• IGMP (Type-P)
• 4-tuple  SrcIPAddr, DstIPAddr, L4P, type

16
Argus Flow Models

• Layer 3
• IPv4 (Type-P)
• 3-tuple  SrcIPAddr, DstIPAddr, L4Protocol   
• Fragments (Type-P1-P2)
• Mapped to any supported Argus flow type.
• Fragments (Type-P)
• 4-tuple  SrcIPAddr, DstIPAddr, L4Protocol, ip_id
• Layer 2
• LLC SNAP Encapsulation (Type-P)
• 5-tuple  SrcMACAddr, DstMACAddr, L3Proto,
  SrcSAP, DstSAP
• ARP (Type-P1-P2)
• 3-tuple ARP_SPA, ARP_TPA, Eaddr
• where the EAddr value is either the SrcMacAddr of
  the REQUEST or the dstMACAddr of the REPLY.
• All other traffic (Type-P)
• 3-tuple  SrcMACAddr, DstMACAddr, L3Protocol

17
Argus Flow Record Format

• Common Type Length Value (TLV) Structure
• Common 16 byte header

struct ArgusRecord    unsigned char type,
cause   unsigned short length   unsigned int
status   unsigned int argusid   unsigned int
seqNumber   union       struct ArgusMarStruct
mar      struct ArgusFarStruct far   
ar_union

• A Start MAR must be the first record in an
  ArgusRecord Stream
• A Stop MAR should be the last record

18
Argus Record Format

• Type
• Type of Argus Record MAR or FAR
• Length
• Length of entire argus record
• Status
• Connectivity status, transition status
• Argus ID
• A unique identifier for the source argus
• Sequence Number
• Management Audit Record (MAR)
• Provides information about argus itself
• Start MAR --- Status MAR ---- Stop MAR
• Flow Activity Record (FAR)
• Provides information about network transaction
  flows that argus track
• The FAR are generated either because of state or
  because of time
• Start FAR transaction started, Stop FAR
  transaction stopped
• Status FAR
• Default time out every 60 seconds

19
Argus Flow Record -MAR
struct ArgusRecord    unsigned char type,
cause   unsigned short length   unsigned int
status   unsigned int argusid   unsigned int
seqNumber    union       struct
ArgusMarStruct mar      struct ArgusFarStruct
far    ar_union
struct ArgusMarStruct     struct timeval
startime, now    unsigned char major_version,
minor_version    unsigned char interfaceType,
interfaceStatus    unsigned short
reportInterval, argusMrInterval    unsigned int
argusid, localnet, netmask, nextMrSequenceNum   
unsigned long long pktsRcvd, bytesRcvd   
unsigned int pktsDrop, flows, flowsClosed   
unsigned int actIPcons, cloIPcons    unsigned
int actICMPcons, cloICMPcons    unsigned int
actIGMPcons, cloIGMPcons    unsigned int
actFRAGcons, cloFRAGcons    unsigned int
actSECcons, cloSECcons    int record_len
20
Argus Flow Record - FAR
struct ArgusFarStruct unsigned char type,
length unsigned short status unsigned int
ArgusTransRefNum struct ArgusTimeDesc time
struct ArgusFlow flow struct ArgusAttributes
attr struct ArgusMeter src, dst
struct ArgusTimeDesc struct timeval start
struct timeval last
struct ArgusFlow union struct ArgusIPFlow
ip struct ArgusICMPFlow icmp struct
ArgusMACFlow mac struct ArgusArpFlow arp
struct ArgusRarpFlow rarp struct ArgusESPFlow
esp flow_union
struct ArgusAttributes union struct
ArgusIPAttributes ip struct ArgusARPAttributes
arp attr_union
struct ArgusIPAttributes unsigned short
soptions, doptions unsigned char sttl, dttl
unsigned char stos, dtos
struct ArgusMeter unsigned int count, bytes,
appbytes
struct ArgusARPAttributes unsigned char
response8
21
Argus Flow Record FAR - Argus Flow
icmp
struct ArgusIPFlow unsigned int ip_src,
ip_dst unsigned char ip_p, tp_p unsigned
short sport, dport unsigned short ip_id
struct ArgusICMPFlow unsigned int ip_src,
ip_dst unsigned char ip_p, tp_p unsigned char
type, code unsigned short id, ip_id
ip
7
8
arp
struct ArgusArpFlow unsigned int arp_spa
unsigned int arp_tpa unsigned char
etheraddr6 unsigned short pad
mac
struct ArgusMACFlow struct ether_header ehdr
unsigned char dsap, ssap
3
4
rarp
struct ArgusRarpFlow unsigned int arp_tpa
unsigned char srceaddr6 unsigned char
tareaddr6
3
esp
struct ArgusESPFlow unsigned int ip_src,
ip_dst unsigned char ip_p, tp_p unsigned
short pad unsigned int spi
6
22
Argus Flow - Canonical Record
struct ArgusCanonicalRecord struct
ArgusRecordHeader ahdr struct ArgusFarStruct
far struct ArgusMacStruct mac union
struct ArgusTCPObject tcp
struct ArgusESPStruct esp struct
ArgusICMPObject icmp struct
ArgusIGMPObject igmp struct
ArgusDHCPObject dhcp struct
ArgusRTPObject rtp struct
ArgusRTCPObject rtcp struct
ArgusARPObject arp struct
ArgusAHObject ah struct
ArgusFRAGObject frag acr_union struct
ArgusAGRStruct agr struct ArgusTimeStruct
time struct ArgusVlanStruct vlan
struct ArgusMplsStruct mpls
struct ArgusMacStruct unsigned char type,
length unsigned short status union
struct ArgusETHERObject ether
phys_union
struct ArgusRecordHeader unsigned char type,
cause unsigned short length unsigned int
status unsigned int argusid unsigned int
seqNumber
struct ArgusETHERObject unsigned char
ethersrc6 unsigned char etherdst6
struct ArgusAGRStruct unsigned char type,
length u_short status unsigned int
count struct timeval laststartime, lasttime
struct ArgusTimeObject act, idle
struct ArgusTimeStruct unsigned char type,
length u_short status struct
ArgusTimeEntity src, dst
struct ArgusVlanStruct unsigned char type,
length unsigned short status unsigned
short sid, did
struct ArgusTimeEntity struct
ArgusTimeObject act, idle
struct ArgusTimeObject int n unsigned
int min unsigned int mean unsigned int
stdev unsigned int max
struct ArgusMplsStruct unsigned char type,
length unsigned short status unsigned int
slabel unsigned int dlabel
23
Argus Flow Record acr union
tcp
struct ArgusTCPObject unsigned char type,
length unsigned short status unsigned int
state unsigned int options unsigned int
synAckuSecs, ackDatauSecs struct
ArgusTCPObjectMetrics src, dst
struct ArgusTCPObjectMetrics unsigned int
seqbase, ackbytes unsigned int bytes, rpkts
unsigned short win unsigned char flags,
pad
dhcp
struct ArgusDHCPObject unsigned int
respaddr
rtp
struct ArgusRTPObject unsigned char type,
length unsigned short status struct
rtphdr src, dst unsigned short sdrop,
ddrop unsigned short ssdev, dsdev
struct ArgusESPObject unsigned int spi,
lastseq, lostseq
esp
struct ArgusESPStruct unsigned char type,
length u_short status struct
ArgusESPObject src, dst
rctp
struct ArgusRTCPObject unsigned char type,
length unsigned short status struct
rtcphdr src, dst unsigned short src_pkt_drop,
dst_pkt_drop
icmp
struct ArgusICMPObject unsigned char type,
length unsigned short status unsigned
char icmp_type, icmp_code unsigned short
iseq unsigned int osrcaddr, odstaddr
unsigned int isrcaddr, idstaddr unsigned int
igwaddr
arp
struct ArgusARPObject unsigned char
respaddr6 unsigned short pad
ah
struct ArgusAHObject unsigned int src_spi,
dst_spi unsigned int src_replay,
dst_replay
igmp
struct ArgusIGMPObject unsigned char
igmp_type, pad unsigned int igmp_group
frag
struct ArgusFRAGObject int fragnum,
frag_id unsigned short status, totlen,
currlen, axfraglen
24
Argus Transport Model

• Record generator (server) supports multiple
  access methods.
• Local storage
• Near-real time record access
• Collector (client) initiated associations
• TCP based control exchange
• Proprietary protocol for capability negotiation
• TCP or UDP based data transfer
• SASL (Simple Authentication and Security Layer,
  RFC 2222) mediated security

25
Access Methods

• Local Storage
• Information Base for Transport Reliability
• Enable retransmission capability
• Support guaranteed delivery
• Provide bulk transfer capability
• Near-Real Time Access
• Push based record transfer
• Integrated management capabilities
• Keep Alive/Heartbeat
• Probe status and state reporting

26
Argus Record Stream

• Collection of Management and Flow Activity
  Records
• Management records convey Argus status/state
  (MAR)
• Flow Activity Records (FAR) convey monitored flow
  state
• Argus Stream/Files have same structure

Start MAR argus Record (required) FAR Argus
Record (optional) . Status MAR Argus Record
(optional) FAR Argus Record (optional) ... Stop
MAR Argus Record (required)
27
Argus Practical Experiences

• Data Model Supports a lot of applications
• Security Assurance
• Detect Service Failure
• Detect DoS attack
• Detect Network Configuration Problem (Policy
  enforcement Validation )
• Accounting/Billing
• Bidirectional Flow Model
• Performance Monitoring in Passive mode (IPPM
  Metrics)
• Connectivity and reachability unidirectional
  and bidirectional
• Packet Loss TCP state machine, sequence number
  tracking logic
• Round-Trip Delay -R option, TCP handshake
  establishment round trip delay metrics are
  provided by default
• Packet Jitter and Jitter variance
• Traffic Management
• Operations Management

28
NG-MON

• Next Generation Network Traffic MONitoring and
  Analysis System
• Developed at DPNM Lab, POSTECH
• Targeting 10 Gbps or higher networks
• To support various analysis applications
• Multimedia streaming conferencing, P2P, game
  traffic analysis
• Network security attack detection and analysis
• SLA monitoring
• Usage-based billing, Customer relationship
  management

29
NG-MON - Requirements

• Distributed, load-balancing architecture for
  scalability
• subdivide monitoring system into several
  functional components
• efficient load sharing between phases and within
  each phase
• pipelined and parallel architecture
• Lossless packet capture
• Flow-based analysis
• aggregate packet information into flows for
  efficient processing
• Considerations for small storage requirements
• Support for various applications

30
NG-MON - Design
Flow Store
Traffic Analyzer
Packet Capturer
Flow Generator
Presenter Web Server
Network Device
User Interface Web browser
analyzed data
stored flows

• NG-MON is composed of 5 phases
• Packet Capture
• Flow Generation
• Flow Store
• Traffic Analysis
• Presentation Reporting

31
NG-MON - Packet Capture
Splitting Device
Network Link
divided raw packet
pkt header messages

• Distribution of raw packets
• by using splitting function provided by an
  optical splitter
• by using mirroring function provided in network
  devices
• Probe
• captures all packets coming into probe
• export buffer-queues one to one with flow
  generators
• fills buffer-queues with packet headers 5-tuple
  based hashing
• collect the scattered packets in the same flow
  into the same buffer-queue

32
NG-MON - Flow Generation
pkt header messages
flow messages

• Distribution of packet header information
• 5-tuple based hashing in the probe
• Packet header messages of potentially the same
  flow get delivered to the same flow generator
• Flow generator receives packet header messages
  and generates flows and exports flow messages to
  flow store

33
NG-MON - Flow Store
Database Query / Response
Traffic Analyzer 1
Traffic Analyzer 2
flow messages
t 2
t 3
Write operations
Read operations

• Separation of write operations from read
  operations
• the destination address of flow message is
  assigned to the flow store according to the time
• While one or more flow stores are inserting flow
  data, the other flow stores are queried by the
  traffic analyzers
• Flow store provides traffic information to
  support various analysis applications
• provides an analysis API to analyzers

34
NG-MON - Traffic Analysis Presentation
Traffic Throughput Analyzer
Flow Store 1
Usage-based billing application
Presenter
DDoS or DoS Attack Analyzer
Other applications

• Analyzer extracts information from Flow Stores
  and can perform application specific analysis
• Separate analyzer is needed for each application

35
NG-MON - Implementation
36
NG-MON - Deployment at POSTECH
http//ngmon.postech.ac.kr
141.223.182.31,32,33,34 POSTECH Computer Center
INTERNET
141.223.182.36 EnterFLEX at Computer Center
141.223.182.40 EnterFLEX at Computer Center
Packet Capture
Flow Generator
Flow Store
Packet Capture
Flow Generator
Analyzer
Presenter
Packet Capture
Flow Generator
Flow Store
Packet Capture
Flow Generator
141.223.182.38 EnterFLEX at Computer Center
1Gbps Optical link
141.223.182.37 EnterFLEX at Computer Center
NetOptics 1Gbps Optical Splitter
POSTECH Gigabit Campus Network
37
NG-MON - Host Data Sent Minute View
38
NG-MON - Host Data Exchanged Minute View
39
NG-MON - Detailed Host Data Received Minute View
40
NG-MON Network Security Analysis Minute View
41
NG-MON Detailed Security Analysis Minute View
42
NG-MON - Application Protocol Minute View
43
NG-MON - Application Protocol Minute View
44
Flow-based Passive Monitoring Tools Summary

• Input L LAN, W WAN, G - Giga
• Measurement A Active, P Passive,
• P Protocol
  distribution, U Utilization, R RTT, L
  Packet Loss
• Scope R Real time, O - Offline