Campus Measurement

1
Campus Measurement

• Matt Zekauskas, matt_at_internet2.edu
• Internet2 Campus Workshop
• Houston, TX

2
Overview

• Measurements Measurement Projects
• Abilene Measurements
• Focus on End-to-End performance and Campus issues

3
We Need Your Help

• I have probably missed something
• Give us the benefit of your experience
• fixing problems
• operating a network
• also join a working group!

4
Measurement Goals

• Solving performance problems
• Network operations
• Network engineering
• Network research
• Operational data
• performance, flows, anomalies
• Network characterization
• how used? Meeting service goals?

5
Measurements from the Center

• Active
• Measurement within Abilene
• Measurement end to end
• Passive
• SNMP stats (esp. core Abilene links)
• Variables via router proxy
• Characterization of traffic
• Netflow OCxMON

6
Measurement Projects

• AMP (round-trip delay, loss, routing)
• moat.nlanr.net/AMP
• 120 Internet2 campuses
• Surveyor (one-way delay, loss, routing)
• www.advanced.org/surveyor
• On many Internet2 campuses (70 sites)
• Abilene presence
• PMA (passive, packet traces)
• moat.nlanr.net/PMA
• 1 min, 8 times a day, 13 sites

7
Measurement Projects

• PingER (round-trip delay, routing)
• http//www-iepm.slac.stanford.edu/pinger/
• Long term data from a few locations to many
• High-energy physics focus
• New iperf data application (bbftp, bbcp) data
• NIMI
• http//www.ncne.nlanr.net/nimi/
• Designed to be platform for experiments
• Undergoing some redesign/revitalization

8
Usefulness

• AMP, Surveyor, Pinger
• If at your campus, a view from your campus
• If at destination, a view of destination
• Look for campus connected to same gigaPoP if not
  at local or destination
• Phase 0 measurement points for e2eperf
• Routing, congestion problems

9
Usefulness

• PMA
• If at your campus, can look at traces for
  anomalies
• Not as useful for on-demand debugging(but dont
  ignore ability to take traces)

10
Surveyor on One Slide

• Continuous measurement
• One-way delay and loss
• 1/sec on Poisson Schedule
• 12 Byte UDP packets
• Traceroutes at 1/600 sec
• 72 Machines
• http//ippm-db.advanced.org/plots/ -- static
  java real-time option
• Currently broken as data repository moves to U
  Wisconsin

11
AMP

• Like Surveyor, but
• Round-trip latency instead of one-way
• Easier to deploy
• Working on more comprehensive set of alarms
• Potentially more available
• Working on a software distribution

12
An Application-Level Example

• Pioneer
• http//pelle.internet2.edu/pioneer/
• Synthesis of existing infrastructure
• Focus video conferencing tests
• Goal use this to tell if video likely to work
• See also
• VideNet Scout
• Beacon being developed at the Ohio ITEC

13
Example Internet2 Detective

• A prototype
• Windows tray application
• Red/green lights, am I on Internet2
• Multicast available
• IPv6 available
• Look for something on/after spring members
  meeting

14
Abilene

• Abilene goal to be an exemplar
• Measurements open
• Tests possible to router nodes
• Web-mediated on-demand measurements
• Throughput tests routinely through backbone
• as well as existing utilization, etc.

15
Abilene Upgrade

• GigE connected high-performance tester
• Latency tester
• Tests that will be available on-demand (web
  mediated)
• Measurements also collected for research

16
Ad-hoc Active on Abilene Today

• Have some OC-3 ATM connected PCs
• With OC-3, can do moderate throughput testing
  (e.g., iperf UDP TCP). 90 Mbps
• Contact me (matt_at_internet2.edu) if you want to
  perform an ad-hoc test

17
Passive - Utilization

• The Abilene NOC takes
• Packets in,out
• Bytes in,out
• Drops/Errors
• ..for all interfaces, publishes internal links
  peering points (at 5 min intervals)
• ..via SNMP polling every 3 sec
• http//hydra.uits.iu.edu/abilene/traffic/

18
(No Transcript)
19
(No Transcript)
20
Passive Characterization

• NetFlow measurements from routers
• QoS
• AS-AS information for K-20 ITN
• http//www.itec.oar.net/abilene-netflow
• http//netflow.internet2.edu/weekly/ (summaries)
• Some sparse via NLANR/MOAT
• http//moat.nlanr.net/PMA/

21
Others via Abilene NOC

• BGP Peering
• MSDP (multicast source discovery) logging
• See http//www.abilene.iu.edu/-gt Monitoring
  Tools

22
End-to-End Measurement Infrastructure vision

• Ongoing monitoring to test major elements, and
  (some, important) end-to-end paths.
• Elements gigaPoP links, peering,
• Utilization
• Delay
• Loss
• Occasional throughput
• Multicast connectivity

23
End-to-End Measurement Infrastructure Vision II

• There are many more paths end to end than can be
  monitored.
• Diagnostic tools available on-demand (with
  authorization)
• Show routes
• Perform flow tests (perhaps app tests)
• Parse/debug flows (a-la tcpdump or OCXmon with
  heuristic tools)

24
Enabling Divide Conquerand Ongoing Monitoring
Wall Jack
P
P
Wall Jack
25
For TCP (and Streaming)

• Eliminating loss is the goal
• Focus on noncongestive losses
• TCP 100 Mbit Ethernet coast-to-coast
• Full size packets need 10-6 Ploss Mathis
• Less than 1 loss every 83 seconds
• http//www.psc.edu/mathis/papers/JTechs200105/
• GigE/655 10-8, 1 loss every 497 seconds

26
Quilt Measurement Project

• A measurement portal
• Example, GPN http//noc.greatplains.net/measureme
  nt/
• SNMP utilization
• Ping data, hops, traceroute
• Iperf server, pchar server
• Router proxy
• TCP tester (options, buffer size, )
• http//www.thequilt.net/measurement/
• (Contact Rick Summerhill, rrsum_at_greatplains.net)

27
Some Commercial Tools

• Caveat only a partial list, give me more!
• Spirent (nee Netcom/Adtech)
• working on a box for end-to-end measurements
• SmartBits test at low high rates, QoS test
  components or end-to-end path
• NetIQ Chariot/Pegasus
• Agilent (like SmartBits, and FireHunter)
• Ixia (like SmartBits/Spirent)
• Brix Networks (like Surveyor, for QoS)
• jaalaM Technologies path debugger

28
Some Noncommercial Tools

• Iperf dast.nlanr.net/Projects/iperf
• See also http//www-itg.lbl.gov/nettest/
• http//www-didc.lbl.gov/NCS/
• Flowscan
• http//www.caida.org/tools/utilities/flowscan/
• http//net.doit.wisc.edu/plonka/FlowScan/
• SLACs traceroute perl script
• http//www.slac.stanford.edu/comp/net/wan-mon/trac
  eroute-srv.html
• One large list
• http//www.slac.stanford.edu/xorg/nmtf/nmtf-tools.
  html

29
If you like living on the bleeding edge

• Matt Mathis just released pathprobe, a Web100
  tool, as a way to test campus networks
  suitability for long-haul TCPs.
• http//www.psc.edu/web100/pathprobe/

30
What You Can Do

• Export SNMP data
• I have an internet2 list, can add you
• Monitor loss as well as throughput
• Performance test point at campus edge
• Netperf or iperf, so can be from anywhere
• Traceroute looking glass
• Commercial (e.g., NetIQ) complements
• Im willing to keep a master list
• Create a measurement portal a-la Quilts

31
A Summer Project

• Measurement box at edge
• Spend month or two with mobile box, checking
  throughput/loss/.. from every point. For
  example, use iperf (or now, pathprobe)
• Eliminate noncongestive losses
• Develop a baseline to get a complete picture of
  the campus map the campus networks

32
What You Can Do

• If you have a Cisco router at your edge, use
  NetFlow and cflowd FlowScan to see your traffic
  characteristics
• RTFM / RMON probes
• See also Joe St.Sauvers presentation from May
  2001 Joint Techs meeting
• http//www.ncne.nlanr.net/training/techs/2001/0514
  /presentations/200105-sauver1.html

33
For TCP Tuning

• Consider Web100 for Linux 2.4
• http//www.web100.org/
• NCNE Tuning Page
• http//www.psc.edu/networking/perf_tune.html
• http//www.ncne.nlanr.net/research/tcp/

34
Contact Information

• Matt Zekauskas, matt_at_internet2.edu
• Measurements Working Group
• http//www.internet2.edu/measurement/
• End-to-end Performance Initiative
• http//www.internet2.edu/e2epi/

35
(Some) URLs

• Http//www.internet2.edu/measurement/
• http//www.advanced.org/surveyor/
• http//moat.nlanr.net/ http//dast.nlanr.net/
• http//www.ncne.nlanr.net/ http//www.ncne.org/
• http//www.caida.org/ http//www.web100.org/
• http//www.auckland.ac.nz/net/Internet/rtfm/
• http//www.slac.stanford.edu/xorg/icfa/ntf/home.ht
  ml
• http//www.merit.edu/ipma/

36
www.internet2.edu