Network Measurement for KREONET -FlowScan-

1
Network Measurement for KREONET-FlowScan-

• 2002.9.26
• ???
• KISTI/KREONET

2
??

• Measurement ??
• FlowScan, FlowScan?
• FlowScan ???
• ?? FlowScan ?? ?? ???

3
Why Network Measurement?

• Network Operators View
• Is the network reliable?
• How is the network used?
• When should the network be upgraded?
• Network Users View
• How much do I use the network?
• Should I invest more or not?
• Does the Service Provider abide by its agreement?

4
Active Measurement

• Method
• inject measurement traffic into the network
• Metric
• Round Trip Time, Packet loss, Topology
• Tools
• Ping, Traceroute, NIMI, Surveyor, PingER, AMP

5
Passive Measurement

• Method
• Do not inject traffic but observe traffic
• Metric
• Link Utilization, Traffic Analysis
• Tools
• Tcpdump, MRTG, FlowScan, CoralReef

6
What is FlowScan?

• A Network Traffic Flow Reporting and
  Visualization Tool developed by Dave Plonka
• FlowScan? CISCO router? ??? flow data? ???? ???
  ??? ???? tool
• FlowScan? ??? ? ?? Perl script module? ??
• a flow collection engine (a patched version of
  cflowd)
• High performance database(Round Robin Database -
  RRD)
• a visualization tool (RRDtool)
• FlowScan? network? border traffic? ???? ???? ??
  ????(5? ??) ???? ??? ??.

7
Hardware

• Netflow version 5? export ??? CISCO router,
  Cisco IOS Release 12.0 and later releases? ??? ??
• Router?? export? flow ??? FlowScan? ??? ?????
  ???? ??? ??? ???? ??
• FlowScan ??? Sparc machine? Solaris, Intel
  machine? GNU/Linux, BSD ? ?? ??
• ??? ?? ??? ??? ?? ??(Dual CPU, ??? SCSI ?? ??? ?)

8
What is flow?

• Packets with the same src ip port, dst ip
  port, protocol
• NetFlow flow information exported by CISCO
  router
• FlowScan uses NetFlow data to analyze the traffic
  data.

9
NetFlow entry
10
FlowScan's Hardware Components
11
Software 1/3 - cflowd

• Original cflowd cflowdmux, cflowd,
  flowcollector
• FlowScan cflowdmux? cflowd ? ??
• Router?? NetFlow version 5? UDP ?? machine? ??
• cflowdmux? ? ??? ?? cflowd? ?? ?
• cflowd? ? ??? ?? ?? ??? ???? ???? ??, 5? ??

12
Software 2/3 FlowScan

• FlowScan.pm ??? Perl script? ??? ??
• FlowScan? cflowd?? ??? flow dump ??? ??? ?
  RRD(Round Robin Datase)? ??
• DB?? Flow? ? ?? ???? ??? ???? ??

13
Software 3/3 RRDtool

• DB? ??? ??? time-series graph? ??
• RRDtool? ?? ?? RRD file? ??? flow? ?? ??? ??
• RRDtool? RRGrapher? GIF? PNG??? ???? graph ??

14
(No Transcript)
15
Existing FlowScan Graph
16
Existing FlowScan Graph (cont.)
17
What is FlowScan?

• Goal Improve FlowScan by attaching query
  interface for detail analysis.
• Motivation
• Lack of traffic measurement tool that supports
  real time visualization and detailed information
  on demand.
• Provide flexibility in analyzing network traffic
  to Network Engineers and Administrator.
• Why FlowScan?
• FlowScan is open source program and provides good
  visualization through the Web, yet does not
  support query interface.
• Who?
• KISTI, KAIST

18
Advantages and Disadvantages

• The Existing FlowScan
• Provides real-time network status graph and set
  of information to show the trend of network
  status and usage.
• More Possible information can be drawn from
  NetFlow data.
• Amount of traffic used by certain host, inter AS
  traffic amount, packet distribution, etc

19
Major Improvement Point from Existing FlowScan

• Using DBMS
• for support flexibility when querying
• MySQL adopted
• Web supported query interface
• More information on traffic data and statistical
  analysis can be obtained by demand.
• ? We named the improved version of FlowScan,
  FlowScan

20
Query interface
21
Predefined query(by user interface)to raw flows

• Total traffic statistic
• All flows in specific time period
• Trace traffic used by specific user
• Protocol statistic
• Port statistic
• As statistic
• Nexthop statistic
• Packet , flow distribution

22
Data Aggregation
netflow
Front table
AS Table
Bypass
Aggregation
...
Port Table
Rawflows
Protocol Table
Top User Table
23
Data Aggregation (contd)

• First, all incoming NetFlow data are inserted to
  front table
• Aggregation module is automatically called every
  15 minute
• After finishing all aggregation, all data in
  front table are moved to raw flows table
• In some aggregation, preserve 90 information but
  only save 20 aggregated data
• Query time is reduced (very much)
• Eventually, old data of raw flows in table
  rawflows will be deleted due to storage
  shortage. But aggregated data will be stored
  forever

24
Problems

• Amount of data (under no sampling on
  KOREN/KREONet2 STAR TAP router)
• 45Mbps ???? 50 ?? usage ?? ?
• ? 115414 flows/5 min, 6MB/5min, 1.7GB/day
• DDoS ???, 3050MB/5min, 10GB/day
• Reporting time more than 1 minute, sometimes
  over 10 minutes
• KISTI and KAIST mending now

25
FlowScan Archetecture
FlowScan
FlowScan
26
FlowScan vs FlowScan

• FlowScan provides
• Traffic analysis by amount of bytes, packets, and
  flows.
• Traffic by IP Protocol, application
• Top inbound/outbound AS
• Top inbound/outbound path AS
• Specific vs Total

• FlowScan provides
• All that FlowScan provides.
• Analysis by desired time period.
• Detailed Information on traffic between ASs
• Nexthop

• One can use FlowScan to see the trend of network
  traffic, and then use FlowScan module to analyze
  certain aspect in detail.

27
Deployment of FlowScan

• KOREN/KREONet2-STARTAP International Link
• 45 Mbps International Link
• http//flowscan.kreonet2.net
• Campus Network-KAIST
• On weather map of KAIST
• http//moran.kaist.ac.kr

28
Traffic From KREONET-STARTAP by Services
(application)
29
Traffic From KREONET-STARTAP Links (by Flows)
2002.1.23 KREONET-STARTAP
30
Using FlowScan to analyze abnormality in the
Network

• Possible detection of DoS attack

31
(No Transcript)
32
(No Transcript)
33
Other Anomalies

• Network Worm Virus
• When there is large portion of sudden smtp
  traffic is shown, one can suspect the possible
  existence of worm virus over the network.
• Code Red, Nimda?
• Port Scanning
• Hacking/Cracking Trials
• Etc..

34
FlowScan ?? Guide

• ??? ?? ??
• FlowScan ??? ??

35
??? ?? ??

• CISCO 7507 ??? IOS 12.0(15)S3
• config terminal
• (config)ip flow-cache timeout inactive 300
• (config)ip flow-cache timeout active 1 ?? ip
  flow-cache active-timeout 1
• (config)ip flow-export version 5
• (config)ip flow-export destination
  150.183.235.100 2055
• (config)ip cef ltdistributedgt //VIP? ?? ????? ?
  ??????? ?? ????? ?
• Ingress interface? ??? ?? ?? ??
• (config)interface Ethernet1
• (config-if)ip route-cache flow

36
FlowScan ?? ?? ??

• FreeBSD 4.3 ?? package ??(?? ??? ??, linux? ??
  ????? package ??? ???? FreeBSD? ?? ???)
• Package ??? /stand/sysinstall -gt Configure-gt
  packages-gtCD-ROM or ftp ?? -gt all or ?? ??
  ??-gtinstall

37
Package Install Screen
38
FlowScan ?? ?? 1

• perl5(???? ???? ??)
• arts-1-1-a8_1
• autoconf-2.13
• GNU bison-1.28
• gmake-3.79.1
• pdksh-5.2.14

39
FlowScan ?? ?? 2

• Cflowd, cflowd patch
• http//net.doit.wisc.edu/plonka/cflowd/?MD
• cflowd-2-1-b1.tar.gz cflowd-2-1-b1-djp.patch ? ??
  ??
• ? ??? ?? ???? ?? ??? ?, ??? ??? ?? patch? ??
• patch ??
• gunzip -c cflowd-2-1-b1.tar.gz tar xf
• cd cflowd-2-1-b1
• patch -p0 lt ../cflowd-2-1-b1-djp.patch
• autoconf optional
• cflowd ?? ??
• ./configure --with-artspp/usr/local
• make
• make install
• ? ?? ?? ?? ?? path? ??
• set path ( /usr/local/arts/bin
  /usr/local/arts/sbin)
• rehash

40
FlowScan ?? ?? 3

• RRD ??
• http//people.ee.ethz.ch/oetiker/webtools/rrdtool
  /pub/
• Package?? ??? source compile? ??
• ?? ??
• gunzip c rrdtool-1.0.33.tar.gz tar xf
• cd rrdtool-1.0.33
• ./configure --enable-shared
• make install site-perl-install
• ? ?? ?? ?? ?? path? ??
• set path ( /usr/local/rrdtool-1.0.33/bin)
• rehash

41
FlowScan ?? ?? 4

• Perl ?? ??
• /stand/sysinstall?? package ??? ??
• p5-Boulder-1.20
• p5-Cflow-1.03
• p5-ConfigReader-0.5_1
• p5-HTML-Table-1.07b
• p5-Net-Patricia-1.010

42
FlowScan ?? ?? 5

• FlowScan-1.006 ??
• http//net.doit.wisc.edu/plonka/FlowScan/
• ?? ??
• ./configure --prefix/usr/flows
• (configure?? rrdtool? ??? error ?? configure ??
  ????ac_cv_path_RRDTOOL_PATH'/usr/local/rrdtool-1.
  0.33' ? ??)
• make
• make -n install
• make install
• mkdir p /usr/flows/graphs
• ? ?? ?? ?? ?? path? ????.
• set path ( /usr/flows/bin)
• rehash

43
FlowScan ?? ?? 6

• clfowd ?? ??
• cp /usr/local/arts/etc/cflowd.conf.example
  /usr/local/arts/etc/cflowd.conf
• vi /usr/local/arts/etc/cflowd.conf
• OPTIONS
• LOGFACILITY local6
• TCPCOLLECTPORT 2056
• PKTBUFSIZE 4000000
• TABLESOCKFILE /usr/local/arts/etc/cflowd
  table.socket
• FLOWDIR /usr/flows
• FLOWFILELEN 1000000
• NUMFLOWFILES 10
• MINLOGMISSED 300

44
FlowScan ?? ?? 7

• cflowd ?? ?? ??
• COLLECTOR
• HOST 150.183.235.100 IP address of
  central collector
• ADDRESSES 150.183.235.100
• AUTH none
• CISCOEXPORTER
• HOST 134.75.20. IP address of
  Cisco sending data.
• ADDRESSES 134.75.20., Addresses of
  interfaces on Cisco
• 210.218.215.,
• 134.75.108.,
• 150.183.2. sending
  data.
• CFDATAPORT 2055 Port on which to listen
  for data.
• SNMPCOMM ' SNMP community name.
• LOCALAS 17579 Local AS of Cisco
  sending data.
• COLLECT protocol, portmatrix, ifmatrix,
  nexthop, netmatrix, asmatrix, tos, flows

45
FlowScan ?? ?? 8

• cflowdmux
• cflowd s 300 O 0 m
• ??? /usr/flows? ip.flows.09 ??? flows.current??
  ??? ??? flows.current? ??? ?? ????? ??
• 5? ?? flows.20010928_0915040900 ? ?? ??? ???
  dump? ??
• ps ax grep flow
• 279 ?? S 000.18 cflowdmux
• 281 ?? S 005.60 cflowd -s 300 -O 0
  m

46
FlowScan ?? ?? 9

• flowscan ?? ?????? /usr/flows/bin ? ?? ??? ??
• CampusIO.cf, flowscan.cf, local_nets.boulder,
  Napster_subnets.boulder
• flowscan.cf
• FlowFileGlob /usr/flows/flows.0-9
• ReportClasses CampusIO
• WaitSeconds 300
• Verbose 1

47
FlowScan ?? ?? 10

• CampusIO.cf
• OutputIfIndexes 2, 9
• LocalSubnetFiles /usr/flows/bin/local_nets.boulder
  
• OutputDir /usr/flows/graphs
• Protocols icmp, tcp, udp
• TCPServices ftp-data, ftp, smtp, nntp, http,
  7070, 554, 1863, 5004
• NapsterSubnetFiles /usr/flows/bin/Napster_subnets.
  boulder
• NapsterSeconds 1800
• NapsterPorts 8875, 4444, 5555, 6666, 6697, 6688,
  6699, 7777, 8888
• ASPairs 00
• TopN 10
• local_nets.boulder
• SUBNET137.68.200.0/24
• DESCRIPTIONour network1
• SUBNET137.68.201.0/24
• DESCRIPTIONour network2

48
FlowScan ?? ?? 11

• flowscan
• ??? ?? ??? ??? ?? ??

49
FlowScan ?? ?? 12

• Save old flows
• mkdir /usr/flows/saved
• mkdir /usr/flows/other
• touch /usr/flows/saved/.gzip_lock
• ??? ??
• cp graphs.mf /usr/flows/graphs/Makefile
• cd /usr/flows/graphs
• gmake
• 554_dst.rrd? ?? ? ??? ???? ??? ??? ?? ???? rrd???
  ???? ??
• rrdtool create 554_dst.rrd --step 300 \
  DSin_bytesABSOLUTE400UU \
  DSout_bytesABSOLUTE400UU \
  DSin_pktsABSOLUTE400UU \
  DSout_pktsABSOLUTE400UU \
  DSin_flowsABSOLUTE400UU \
  DSout_flowsABSOLUTE400UU \
  RRAAVERAGE01600 \ RRAAVERAGE06600 \
  RRAAVERAGE024600 \ RRAAVERAGE0288732 \
  RRAMAX024600 \ RRAMAX0288732

50
FlowScan ?? ?? 13

• crontab ??
• FlowScan stuff
• make the graphs
• 0,5,10,15,20,25,30,35,40,45,50,55 test -f
  /usr/flows/graphs/Makefile cd
  /usr/flows/graphs /usr/local/bin/gmake -s
  gt/dev/null
• copy files in internet directory
• 3,8,13,18,23,28,33,38,43,48,53,58 cp
  /usr/flows/graphs/.png /usr/local/webdocument/
  cp /usr/flows/graphs/.html /usr/local/webdocum
  ent/
• gzip the saved flow files
• 2,7,12,17,22,27,32,37,42,47,52,57 test -d
  /usr/flows/saved cd /usr/flows/saved
  /usr/flows/bin/locker -ne .gzip_lock
  "/usr/local/bin/ksh -c '/bin/ls
  flows.0-9!(.gz) 2gt/dev/null /usr/bin/xargs
  -n1 /usr/bin/gzip'"
• Purge the flow files
• find(1) -mtime 1 was insufficient - I want to
  delete them as soon as they're
• n' hours old
• 0 /usr/bin/find /usr/flows/saved -type f
  -name 'flows.' -print /usr/bin/perl -e 'now
  time seconds 286060 while (ltgt) chomp
  (_at__ stat _) (now - _9 gt seconds)
  print _, "\n" ' /usr/bin/xargs /bin/rm -f

51
FlowScan ?? ?? 14

• vi /usr/local/etc/apache/httpd.conf
• DocumentRoot /usr/local/webdocument
• ScriptAlias /cgi-bin/ "/usr/local/webdocument/cgi-
  bin/"
• ltDirectory "/usr/local/webdocument/cgi-bin/"gt
• apachectl start
• ????? ??? flowscan? ?? ???? ??? ?? ??.
• FlowScan? ?? ?? ???? ??? ??.

52
FlowScan ?? ??

• ?? ?? ?? ?
• optimization, visualization, ?? ?? ?? ?? ?? ??
• 2002? ??? ?? ?? ??
• test? ??? ?? ??? ?? flowscan.kreonet2.net? ?????
  ?? ??? ?? ?? ??.

53
??

• FlowScan? ?? ??? passive measurement ?? ??
• ?? ?? FlowScan? ?? ? ??? ??? ??? ???
• KREONET? ???? ??, FlowScan, FlowScan ?? ?? ?? ?
  ?? ??
• Contact ??? mhlee_at_kisti.re.kr

54
References

• KREONET FlowScan http//flowscan.kreonet2.net
• KAIST Project Homepage(developers)
  http//cosmos.kaist.ac.kr/navihp
• KAIST FlowScan Homepage
• http//moran.kaist.ac.kr
• http//net.doit.wisc.edu/plonka/lisa/FlowScan/
• http//www.caida.org/tools/measurement/cflowd/