InterDomain Services

1
Inter-Domain Services

• Presented to Quilt Spring Workshop
• Salt Lake City, UT
• By
• Linda Winkler
• winkler_at_mcs.anl.gov
• April 4, 2007

2
Motivation
Emerging large-scale, globally distributed
applications require more sophisticated network
services than have previously been delivered.

• Support applications that demand capabilities
  that are hard to support in a shared packet
  infrastructure
• Large bandwidth applications
• Applications dont want to worry about their
  impact on other network users, or vice versa
• Applications that benefit from circuit
  characteristics, and that may be low bandwidth in
  nature
• Dedicated network resources
• Deterministic performance
• Repeatable and predictable
• Very high performance
• Multi gigabit flows, low latency, low loss,
  minimal jitter
• User community expectations
• Extreme bandwidth, dedicated and on-demand
• Inter-regional and international connectivity
  requirements
• Reserve and schedule network resources in advance
• In conjunction with availability of non-network
  resources (e.g. instruments, clusters, viz
  devices)
• Flexible and dynamic
• Capability to acquire resources on short notice
  from many potential service/resource providers
• E-Science services growth
• Broad scope of applications

3
TrendsgtSet the Stage

• Applications community desire to treat network
  resources as an integral Grid resource
• RE Optical Networks
• Locally-managed fiber termination points
• Locally-organized peering relationships
• Locally-controlled layer-0/1/2/3 services
• Community ownership and control of network
  infrastructure
• Ubiquitous Ethernet
• Most-requested client service interface
• Both point-to-point and virtual-LAN topologies
• Apps consuming bandwidth in Ethernet-sized
  increments
• 10G has become a commodity
• Application expectations for error-free
  connections keep rising. Apps with minimal error
  checking running on pipes with almost no error
  correction, across inter-RON pipes with no error
  reporting ? a recipe for failure
• Convergence Optical Ethernet Layers
• Ethernet client interface, Optical transport
• Optical service established via GMPLS
• Control plane technologies emerging to help meet
  user expectations in the current environment
• FPGA technology
• Allowing WDM transport hardware to be built with
  software defined functionality
• Flexibility, fewer components, configurations,
  spares

4
Technology Choices

• Reasons you may want/need build circuits
• Dedicated bandwidth connections for deterministic
  file transfers
• Dedicated bandwidth low jitter for instrument
  control or interactive applications
• Connector backhaul to your IP Network
• Traffic engineering of your IP Network
• Dynamic router-to-router circuits for traffic cut
  through
• Computer to Computer communications
• Processor to memory or block data storage system
  access
• Setting up application specific topologies to
  create optimize distributed application or data
  storage systems
• There are many technologies available over which
  to construct these circuits
• IP router-based Multiprotocol Label Switching
  (MPLS) Label Switched Paths (LSPs) circuits
• Ethernet VLAN based circuits
• SONET/SDH TDM circuits
• Wavelength Division Multiplexing (WDM) circuits
  

5
Mapping Technology to Services
Network Technology
Service
IP
Layer 3
IP
Ethernet
Layer 2
Ethernet VLAN
SONET DM encap GFP
Lightpath (sub-lambda)
Layer 1
Lambda
Wavelength
Layer 0
Fiber
Dark Fiber
6
Topology for L1 Dynamic Optical Multicast Testbed
OC192/STM64 10GE LAN 750-980 Mbps

• 750-980 Mbps Hi-Def Video
• 3x12 Optical Multicast StarLight OME

10 G WAN
Nortel
UvA
Nortel HDXc
iCAIR
Nortel HDXc
3x750 M
3x750 M
3x12 Optical multicast
UvA
iCAIR
iCAIR
Nortel
Nortel
7
Topology for L2 Multicast Demo

• 1.5 Gbps Hi-Def video
• 3x12 Ethernet multicast StarLight SCinet
• Layer1 and Layer2 networking

Baton Rouge (LSU Class)
Brno
Tampa
2
1
1
OMNInet
2
2
OMNInet
4
3 x 1.5G
2
OPTERON
3x1.5G
2
3x1.5G
1
1
Brno
3
Tampa
1
LSU
2
2
1
LONI
Tampa
OPTERON AMD PC
CAT 6504
Brno
LONI
8
Unification of Data Plane Technology Across
Layers 1/2/3One Method
Alignment of VLANS
IP Network
Circuit Switched
MPLS tunnel
VLAN over MPLS
Layer-1/2 connection
VLAN over Ethernet or SONET
9
Dynamic Circuit Setup/Allocation

• Lots of work in the community on dynamic resource
  allocation (lightpaths) and how were going to
  get there from here, eg GLIF forum.
• Soon we will be to able to rely on
  multiprovider/multivendor control plane signaling
  for circuit establishment.
• Simplify or add complexity?
• Dynamic resource mechanisms will increase our
  capabilities and enable new capabilites, like
  timesharing resources between large
  short-duration flows.
• Cost TBD

10
Connection Brokering per Domain

• Network Descriptions
• Paths through network
• Information about paths
• Common Services Definition
• Description of connectivity service
• Predictable
• Verifiable
• Repeatable
• End-to-end
• Agreement between user and provider(s)

11
Control Plane Functions

• Bandwidth required
• Allow for reservation of future bandwidth
• User authentication
• Report status of infrastructure
• Retain and share information on topology
  resources and reachability
• Routing- topology discovery and path computation
• Signaling- standard communications protocols
  between network elements for the establishment
  and maintenance of connections
• Neighbor discovery- cross-domain sharing of
  connectivity details among neighbors
• Domain (local) resource management- accounting of
  available resources

12
Interdomain Connection Setup
Interdomain Control Plane
TDM (SONET/SDH Layer)
Control Plane
Data Plane
TDM or L2 substrate
RON / NREN
13
Challenges

• Engineering/configuration/setup
• OM complexity
• Troubleshooting
• Fault managment
• Monitoring
• Performance
• Debugging tools
• End-to-end measurement for monitoring and
  performance analysis
• Policies
• a way to enforce policies and filters at the
  domain boundaries to process the incoming
  inter-domain requests based on certain agreed
  trust and service levels/contracts between
  domains
• Authentication, Authorization, Accounting (AAA)
• Security
• Scheduling
• Service Level Agreements (SLAs)
• User interface
• Capacity planning
• Many lightpaths traverse a number of transport
  boxes and domains
• Number of domains participating on the rise
  (backbone providers, RONs, campuses). Are they
  prepared for the challenge?

14
Dynamic Resource Allocation Research

• MAX DRAGON
• Internet2 Circuit Services- based on extension of
  the DRAGON control plane architecture
• Internet2 BRUW (Bandwidth Reservation for User
  Work)
• ESNet Science Data Network and the OSCARS
  (On-demand Secure Circuits and Advance
  Reservation System) project
• DANTE/GEANT JRA3 project
• SURFnet and collaboration with Nortel on the DRAC
  project
• University of Amsterdam, Network Description
  Language
• European Union PHOSPHORUS Project
• G-Lambda project (Japan)
• CANARIE- UCLP
• DOE UltraScience Network
• DOE Hybrid Multi-Layer Network Control Project
  (ESnet, ORNL, I2, ISI East, Tenn Tech)
• TeraPaths Brookhaven National Lab
• Enlightened Computing Project (NCREN, MCNC, LSU,
  NCSU, RCI)
• Cheetah (Univ of Virginia)
• UltraLight
• DICE (DANTE, Internet2, CANARIE, ESnet)
• OptIPuter/iCAIR

15
Enlightened Computing Testbed
To Asia
To Canada
To Europe
SEA
POR
BOI
EnLIGHTened wave (Cisco/NLR)
CAVE wave
PIT
OGD
DEN
CHI
KAN
CLE
SVL
WDC
Cisco/UltraLight wave
LONI wave
TUL
DAL

• International
• Partners
• LUCIFER - EC
• G-Lambda - Japan
• GLIF

• Members
• MCNC GCNS
• LSU CCT
• NCSU
• (Subcontract) RENCI

HOU

• Official Partners
• ATT Research
• SURA
• NRL
• Cisco Systems
• Calient Networks
• IBM

• NSF Project Partners
• OptIPuter
• UltraLight
• WAN-in-LAB
• DRAGON
• Cheetah

16
HD Media Testbed to support class on
Computational Science
17
Enlightened Architecture
18
Internet2 Circuit Service Attributes

• Physical connection
• 1G or 10G Ethernet
• OC-48 or OC-192 SONET
• Guaranteed bandwidth (STS-1 to STS-192)
• Circuit Service Type
• Point-to-point Ethernet framed SONET Circuit
• Point-to-point SONET Circuit
• Deterministic behavior (bounded jitter, latency
  and packet loss)
• Static or dynamic

19
NLR

• FrameNet
• Ethernet-based transport services over the
  nationwide NLR optical infrastructure
• National Exchange Fabric
• Dedicated FrameNet
• Non-Dedicated FrameNet
• Research with MCNC
• Calient Optical Switch

20
References

• dragon.maxgigapop.net
• www.es.net/oscars
• www.internet2.edu/network/library/internet2-networ
  k-service-descriptions-02272007-v1.1a.pdf
• www.ist-phosphorus.eu/
• www.mfaforum.org/interop/GMPLSwhitepaper_Final1009
  021.pdf
• www.glif.is
• enlightenedcomputing.org
• www.oiforum.com
• IETF
• ITU