Networking Research Overview

1
Networking Research Overview

• Micah Beck
• Assoc. Prof., Computer Science
• Director, LoCI Laboratory
• University of Tennessee
• SciDAC PI Mtg 24 March 2004

2
SciDAC Networking Research Projects Goals

• Goal Phase I
• Develop data movement tools and infrastructures
  to support real-time data-intensive SciDAC
  applications
• To develop advanced network tools enable SciDAC
  applications efficiently measure, predict, and
  diagnose end-to-end performance (2 projects)
• To develop and deploy cyber security tools to
  support group collaborations in grid
  infrastructures
• Goal Phase II
• Deploy the advanced tools developed in phase I in
  production infrastructures to support network
  intensive SciDAC projects

3
Logistical Networking Tools, Applications
Architecture

• Micah BeckJack DongarraJames S. Plank
  University of Tennessee
• Rich Wolksi
• University of California,Santa Barbara
• http//loci.cs.utk.edu/scidac

4
Project Thrusts

• Dongarra Application Development
  Tools/Environments
• NetSolve/GridSolve
• Wolski Network Monitoring/Prediction
• Network Weather Service
• Beck Plank Logistical Networking
  Infrastructure, Middleware Support
• Internet Backplane Protocol
• Logistical Runtime System

5
Internet Backplane Protocol

• Overlay intermediate node providing services
  based on enriched resources
• Storage file system, RAM, disk
• Transfer TCP (std, compressed), UDP(SABUL,
  mcast), SAN/WAN
• Processing primitive operations (alpha)
• 100s of IBP depots deployed worldwide
• 1.4 alpha release persistent sockets optional
  authentication, usage logging

6
Logistical Networking Tools

• Logistical Runtime System (LoRS)
• E2E Services Fault tolerance (Reed-Solomon),
  encryption (AES), compression, high perf. data
  movement strategies
• Library, command line, GUI, Web tools
• Ported to all compute platforms (Cray OS
  problems)
• Logistical Backbone (L-Bone)
• depot monitoring, resource discovery
• Logistical Distribution Network (LoDN)
• directory services, content distribution
• Java Web Start delivery of tools

7
SciDAC Application Impact

• Terascale Supernova Initiative(A. Mezzacappa,
  ONRL J. Blondin, NCSU, D. Swesty, SUNY Stony
  Brook)
• Five 1.6TB depots deployed at TSI sites
• Energy Fusion Research (S. Klasky, PPPL)
• Depots deployed on PPPL cluster nodes
• Dataset transfers O(1TB) _at_ 1-400 Mb/s
• Simulations at NERSC and ORNL
• Control/viz at ONRL, NCSU, Stony Brook, PPPL
• Transfers span ESNet, Abilene
• CS/Physics collaboration, science getting done!

8
TSI Site Deployment ORNL, NCSU, SUNY Stony
Brook, NERSC, UCSD
9
SciDAC Technology Impact

• Spanning heterogeneous networks
• Ultrascale (10 Gbps) wide area transfers require
  specialized systems
• Optically swtiched networks (e.g DOE Science
  UltraNet) do not peer with IP
• Serving scalable communities
• Staging and caching at intermediate nodes
• Processing data in transit
• Common services on distributed data

10
Transit Networking Architecture
Application
Transport

Network
IP
common interface
Transit
link
Local
Physical
transfer
processing
storage
11
INCITE Edge-based Traffic Processingfor
High-Performance Networks
R. Baraniuk, E. Knightly, R. Nowak, R. Riedi
Rice University L. Cottrell, J. Navratil, W.
Mathews SLAC W. Feng, M. Gardner LANL web site
incite.rice.edu
12
INCITE Project

• InterNet Control and Inference from The Edge
• on-line tools to characterize and map host and
  network performance as a function of time,
  space, application, protocol, and service

13
INCITE Thrusts and Tools

• Thrust 1 Multiscale traffic analysis and
  modeling techniques
• wavelet, multifractal, connection-level models
• Thrust 2 Inference and control algorithms for
  network paths, links, and routers
• end-to-end path probing and modeling
• network tomography and topology discovery
• advanced high-speed protocols
• Thrust 3 Data collection tools
• active measurement infrastructure
• passive application-layer measurement

14
pathChirp

• Goal
• estimate instantaneous available bandwidth (ABW)
  on an end-to-end network link
• Basic probing paradigm
• stream packets at some rate
• no queuing delay ? rateltABW
• queuing delay builds up ? rategtABW
• Until now tradeoff
• high accuracy has required high volume probing
  (inefficient)
• Unique to pathChirp
• variable rate probe packet train (exponentially
  spaced chirp)
• 10x more efficient than competing techniques

15
Network Tomography
From end-to-endmeasurements
infer internal topology and delay/loss
characteristics
16
TCP - Low Priority

• Goal
• utilize excess bandwidth in a non-intrusive
  fashion
• Methodology
• sender-side modification of TCP delay-based
  approach
• Applications
• bulk data transfers
• available bandwidth monitoring
• P2P file sharing
• High-speed TCP-LP
• TCP-LP HSTCP
• implementation
• Linux-2.4.22-web100
• experiments
• Stanford - Ann Arbor
• Stanford - Gainesville

• TCP alone 745.5 Kb/s
• TCP plus 739.5 Kb/sTCP-LP 109.5 Kb/s
• TCP-LP is invisible to TCP

17
Changes in network topology (BGP) can result in
dramatic changes in performance
Hour
Samples of traceroute trees generated from the
table
Los-Nettos (100Mbps)
Remote host
Snapshot of traceroute summary table
Note 1. Caltech misrouted via Los-Nettos 100Mbps
commercial net 1400-1700 2. ESnet/GEANT working
on routes from 200 to 1400
Drop in performance (From original path
SLAC-CENIC-Caltech to SLAC-Esnet-LosNettos
(100Mbps) -Caltech )
Back to original path
Dynamic BW capacity (DBC)
Changes detected by IEPM-Iperf and AbWE
Mbits/s
Available BW (DBC-XT)
Cross-traffic (XT)
Esnet-LosNettos segment in the path (100 Mbits/s)
ABwE measurement one/minute for 24 hours Thursday
9 October 900am to Friday 10 October 901am
18
Crossing the Application/Network Divide
Send data over network
Application
Segmentation
TCP
Flow Congestion Control

• Implications to the
• application?
• Insights for high-
• performance network
• protocols?

Checksums
IP
Fragmentation

Data Link
Network
Network monitors focus here.
19
TICKET and MAGNETMUSETICKET Traffic
Information-Collecting Kernel with Exact
TimingMAGNeT Monitor for Application-Generated
Network TrafficMUSE MAGNET User-Space
Environment
Send data over network
Application
Segmentation
TCP
Flow Congestion Control
Checksums
IP
Fragmentation

Data Link
Network
For more information, go to www.lanl.gov/radiant/p
ubs.html
20
MAGNeT ? MAGNET Monitoring Apparatus for General
kerNel-Event Tracing (at nanoscale granularity)

• Why not extend monitoring to kernel events in
  general? Software Oscilloscope for Cluster and
  Grids
• Debugging
• e.g., IdentifiedLinux OS bug in the scheduler for
  SMPs.
• Can be used to deploy, debug, and monitor the DOE
  UltraNet (UltraScienceNet), e.g., dynamic
  provisioning.
• Performance Optimization
• Improved performance of 10GigE adapters by 300.
  Can improve end-to-end performance of DOE
  UltraNet.
• Monitoring Grid Applications
• Integrated MAGNET with SciDACs PERC TAU and
  SciDACs PERC SvPablo/Autopilot.
• Adaptive Resource-Aware Applications
• SciDAC Deployment PERC, Supernova Science Ctr,
  Transit Network Fabric Terascale Supernova
  Initiative Fusion Energy (emerging), and Earth
  Systems Grid II (emerging).

For more information, see M. Gardner, W. Deng,
T. Markham, C. Mendes, W. Feng, and D. Reed, A
High-Fidelity Software Oscilloscope for Globus,
GlobusWorld 2004, Jan. 2004.
21
Bandwidth estimationmeasurement methodologies
and applications

• k claffy (CAIDA),
• Constantinos Dovrolis (Georgia Tech)

22
Project goals

• Develop estimation techniques and public-domain
  tools for the estimation of end-to-end
• Network capacity (bottleneck bandwidth)
• Available bandwidth (residual capacity)
• Focus 1 non-intrusive, fast, and accurate
  techniques
• Focus 2 high-bandwidth paths (up to 1Gbps)
• Compare and validate different tools in
  reproducible and realistic net conditions
• Apply bandwidth estimation in transport and
  overlay routing problems
• Disseminate research results at conferences and
  journals

23
Main accomplishments

• Pathrate capacity estimation tool
• Based on packet pairs and trains
• Publication Transactions on Networking, to
  appear in 2004, and Infocom 2001
• Pathload available bandwidth estimation tool
• Based on self-loading periodic streams
• Publications at ACM SIGCOMM02 and PAM 2002
• Both tools are available at
• www.pathrate.org
• About 200 downloads per month (and increasing)
• Able to measure up to 1Gbps paths, even in the
  presence of interrupt coalescence
• See publication at PAM 2004
• 1st Bandwidth Estimation workshop at CAIDA, Dec03

24
Main accomplishments (cont)

• Created testbed at CAIDA with several high-bw
  routers and switches and realistic cross traffic
• Tested all existing open-source bandwidth
  estimation tools
• Showed that, despite that several such tools
  exist, very few are accurate and consistent
• Developed estimation technique for passive
  capacity estimation
• See publications at IMC 2003 and PAM 2004
• Showed that per-hop capacity estimation tools
  (pathchar-like) are not accurate in the presence
  of layer-2 switches
• See publication at Infocom 2003
• Created ANEMOS, a distributed system for
  automated on-line monitoring of many network
  paths
• See publication at PAM 2003

25
Ongoing work

• Created SOBAS, an automatic socket buffer sizing
  technique based on available bandwidth estimation
• Basic idea limit TCP window based on available
  bandwidth before the connection causes losses
• Does not require changes in TCP
• Develop estimation technique for the variation
  range of available bandwidth in different time
  scales
• Variation range is crucial for some applications,
  including overlay routing
• Evaluate the predictability of available
  bandwidth process in Internet traffic
• How far in the future can we predict the avail-bw
  with a given accuracy?
• Use of bandwidth estimation in overlay network
  routing and in UltraScienceNet dynamic optical
  circuit bandwidth provisioning

26
Security and Policy forGroup Collaborationhttp/
/www.mcs.anl.gov/dsl/scidac/security/

• PIs
• Steven Tuecke (ANL)
• Carl Kesselman (USC/ISI)
• Miron Livny (U. Wisconsin)
• Technologies involved
• Globus Toolkit
• Condor

27
Problem

• Scalable, fine-grain policy management for
  large, dynamic collaborations
• Large number of individually managed resources,
  each with own policies
• Large number of users
• Users and resources in different domains
• Community policies on use of resources

28
Goals of this Project

• Design, develop and standardize tools for
  maintaining structure of a collaboration
• Take into account collaboration policy, user
  privileges, site policies, resource policies,
  etc.
• Improve significantly the integration of local
  security environments
• E.g., Kerberos
• Instantiate our research results into a framework
  that makes it useable to a wide range of
  collaborative tools
• Globus Toolkit, Condor
• Work within standards community to socialize and
  standardize our approaches
• GGF, IETF, OASIS

29
Our Process
Engage with communities
Get feedback
Design and develop solutions
Integrate into community software
Standardize solutions for greater acceptance
Evaluate and guide emerging standards
30
Delivered Solutions

• Fine-grained Policy RD
• Community Authorization Service
• Dynamic Policy Reconciliation
• Site Security Integration
• KCA/Kx509
• Authorization Callouts
• Grid Security Usability
• SimpleCA /Online CA / MiniCA
• Online Credential Repository

31
Standards and Implementations

• X.509 Proxy Certificates
• GSSAPI extensions
• Policy work SAML, XACML
• Policy Reconciliation

CAS