OnDemand eScience Grids based on AdaptiveMesh Network Architecture

1
On-Demand e-Science Grids based on Adaptive-Mesh
Network Architecture

• NEP Virtual Organization
• - Canadian Light Source Inc. (CLS)
• - Optimum Communications Services, Inc. (OCS)
• - TRLabs
• - CYBERA
• Mark Sandstrom, President, OCS
• mark_at_optimumzone.net

2
Adaptive-Mesh for On-Demand E-Science Grids

• The goal To enable dynamic, high-performance and
  cost-efficient (i.e. practical)
  location-independent scientific collaboration
• Pilot applications to connect CLS users, CYBERA
  computing facilities and TRLabs member
  universities via a realtime self-optimizing,
  self-organizing Adaptive-Mesh grid network
• Examples of current CLS projects demanding
  bursty, yet high-performance network
  connectivity
• Protein crystallography (CMCF)
• Materials sciences (VESPERS)
• Need to dynamically integrate cross-disciplinary
  virtual labs over WANs
• ? Need For On-Demand E-Science Grid Networks

3
The Platform

• Applications
• CLS, CLS users, TRLabs member universities
• Innovative grid technologies
• OCS Adaptive-Mesh
• E.g. UCLP based way to form Adaptive-Mesh groups
  on-demand
• Network and computing facilities
• TRnet
• CYBERA (super computing and data storage)

4
Proof of Concept Adaptive-Mesh over TRnet
Adaptive-Mesh networks can extend to campus
networks and beyond to frequent (intl)
collaborators
Saskatoon - CLS Synchrotron facility
Edmonton - Univ. of Alberta nanotech research -
OCS Adaptive-Mesh remote management - TRLabs
TRnet remote management
Adaptive-Mesh grid for remote, on-demand
scientific collaboration
Winnipeg - Univ. of Manitoba medical
research with CLS using high bandwidth
remote imaging
Winnipeg - TRLabs / Univ. of Regina e.g. high
bandwidth remote imaging (new digital media
development)
Calgary - TRLabs / Univ. of Calgary e.g.
environmental, geological research done with
CLS - CYBERA super computing facilities
5
CLS Remote User Network Requirements

• General characteristics (per CLS)
• Secure data transfer
• Real-time performance Guaranteed QoS
• ? Public Internet not a feasible network solution
  
• Users located at research universities,
  institutes and customer corporate facilities
  across Canada and internationally
• ? Purpose-built networks too inflexible and
  costly
• ? Need for a secure, deterministic, flexible and
  cost-efficient E-Science Grid
  network

6
Saskatoon
MPLSrouter 1.B
MPLSrouter 1.A
Assume each site has a pair of mutually
protecting MPLS edge routers
Requirements for the network grid 1) Provide
maximum possible amount of non-blocking,
high-availability, direct light path like
inter-site connectivity among the five sites
using the single 10Gbps wavelength of TRnet. 2)
10Gbps BW on-demand between any two sites. 3)
Network connectivity to be provided as a
transparent managed service, and must be
administration-free for the users! Rationale The
network must enable transparent, on demand
scientific collaboration, instead of requiring
the users to spend resources on network
administration
Edmonton
Winnipeg
MPLSrouter 5.B
MPLSrouter 2.A
MPLSrouter 5.B
MPLSrouter 2.B
Regina
Calgary
MPLSrouter 4.A
MPLSrouter 4.B
MPLSrouter 3.A
MPLSrouter 3.B
7
Network Implementation Approaches

• Packet ring (routers/switches directly on the
  10Gbps ring)
• Hop count between access sites up to 4
• Unrelated traffic streams competing for the same
  capacity
• ? Traffic that will not even get through on time
  can block other traffic
• ? Non-deterministic performance, potentially
  high latencies and packet loss rate, causing low
  actual efficiency
• Hub and spoke
• Requires core L2/3 switches/routers, which need
  to be configured by a 3rd party service provider,
  and are not transparent
• ? Not an administration-free, transparent
  network as required
• Requires 10 SONET ADMs in addition to the 2 core
  L2/L3 switches/routers ? in
  practice also cost prohibitive
• Adaptive-Mesh
• 4xOC-48 access per site, totaling 50Gbps,
  possible with 10 OCS ITN nodes
• No core routers/switches and no ADMs
• Transparent packet forwarding controlled directly
  by customers edge routers

8
Preferred Network Implementation

• Packet ring or hub-and-spoke cannot even in
  theory provide any more access capacity than
  Adaptive-Mesh (i.e. 50Gbps on the 10Gbps) ring
• The architectural advantages of Adaptive-Mesh,
  i.e., maximized throughput with deterministic
  performance and built-in security, will become
  even more compelling when building larger
  on-demand e-science grids
• ? Proposal to trial Adaptive-Mesh on TRnet, to
  dynamically connect TRLabs member universities
  and CLS as a pilot project of the architecture
• ? After such a successful pilot, Adaptive-Mesh
  should be considered also for wider deployment on
  research network backbones
• ? These concepts of flexible, cost-efficient,
  high-performance and secure on-demand grids (and
  grids of grids) can form the basis of the Future
  Internet

9
Saskatoon
Adaptive-Mesh Grid over TRnet Implementation Archi
tecture
MPLSrouter 1.B
MPLSrouter 1.A
ITN IF Modules
IM 1.A
IM 1.B
ITN Adaptive-Mesh provides direct Layer 1
circuits (of adaptive bandwidth) between each
pair of A IMs, as well as B IMs, with only
one instance of packet-forwarding between the
MPLS router interfaces See next slide for
network ring capacity requirements of an
Adaptive-Mesh, e.g. between the A IMs of this
network ?
Edmonton
Winnipeg
MPLSrouter 5.B
MPLSrouter 2.A
IM 2.A
IM 5.B
MPLSrouter 5.A
MPLSrouter 2.B
IM 2.B
IM 5.A
Single 10G wavelength ring sufficient for the
twenty 2.5G (OC-48c) access points i.e. 50 Gbps
of MPLS packet-switched access capacity
IM 4.B
IM 3.A
MPLS over OC-48c
IM 4.A
IM 3.B
Regina
Calgary
MPLSrouter 4.A
MPLSrouter 4.B
MPLSrouter 3.A
MPLSrouter 3.B
10
IM 1
All ITN IF Modules (IMs) able to map packets on
all AMBs passing by them an AMB carries packets
to its destination IM from all source IMs along
them.
Each Adaptive-Concatenation Mux Bus (AMB)
capacity MxSTS-1
WDM/SONET cloud
East AMB to IM 1
West AMB to IM 1
West AMB to IM 2
East AMB to IM 5
IM 2
IM 5
As many AMBs as there are IMs in the local
ring-half, i.e. (N-1)/2 AMBs for N-node
full-mesh, needed in each ring direction. In
this case, N5 and thus (5-1)/2 2 AMBs needed
per ring direction. Thus, protected, non-blocking
full-mesh using AMBs among N nodes with MxSTS-1
worth of IF capacity to the network per node,
requires (N-1)/2xSTS-M of ring capacity. For
example, (5-1)/2xSTS-482xSTS-48 ? two 5-node
2xSTS-48 rate A-Ms possible over STS-192 ring.
East AMB to IM 2
West AMB to IM 5
Each IM interfaces with a customer router, or IM
of another Adaptive-Mesh, over a pair of PPP
links, each with nominal capacity of MxSTS-1,
e.g. OC-48c for M48.
West AMB to IM 3
East AMB to IM 4
IM 3
East AMB to IM 3
West AMB to IM 4
IM 4
MPLS LER (site 4)
11
Adaptive-Mesh Demo

• The Worlds First, Realtime
• Self-Optimizing, Self-Organizing
• Network In Action...
• Geoff Kliza, VP Operations, OCS
• geoff_at_optimumzone.net

12
(No Transcript)
13
Double AMB Test Network Configuration

• 5 ITN nodes on OC-48 ring
• Test requires 2 x OC-12 AMBs
• Nodes 1, 2, 3 and 4 are sources to Node 5
• The AMB under test carries traffic generated by
  Port 1 and Port 2 of Agilent N2X tester

14
Test Setup

• Since Adaptive-Mesh architecture of ITN is formed
  of multiple similar, though independently
  operating AMBs, only a single AMB needs to be
  tested to fully characterize ITN Adaptive-Mesh
• Agilent N2X MPLS router tester provides priority
  and bulk traffic of random packet sizes from
  64-1500 bytes
• Each of the four source-destination traffic
  streams periodically peaks up to 100 of the
  OC-12c link capacity
• AMB bandwidth allocation among the sources
  continuously optimized to maintain maximum usage
  of destination egress capacity
• Direct circuit like QoS is maintained even at
  100 traffic loads

15
Adaptive-Mesh for On-Demand E-Science Grids...
and the Future Internet

• Network to enable, instead of restrict, on-demand
  scientific collaboration
• Dynamic access and sharing of scientific,
  computing, storage etc. facilities
• ? The more broadly Adaptive-Mesh is deployed in
  network backbones, the more practical and
  cost-efficient worldwide e-Science becomes
• A need to dynamically configure Adaptive-Mesh
  groups among sites with higher mutual network
  throughput requirements, e.g. based on UCLP
• The longer-term vision
• Internet like flexibility and reach, with direct
  light path like performance

16
We look forward to working with you! Optimum
Communications Services, Inc. www.optimumzone.net

17
Context - Connectivity Requirements for Core
Network
Content site / Data center SAN
All MPLS backbone network access points
interconnected as if through a dedicated
point-to-point L1/0 connections to the doubled
core MPLS switches dedicated for the mobile
operator
. . .
. . .
PSTN
LER
LER
LER/ MGW
Internet
LER/ BGR
PSTN
. . .
Core MPLS switch (B)
Core MPLS switch (A)
Virtual core routers/switches transparently
interconnecting all the LERs
MGW
Internet
LER
BGR
. . .
Wireless access
LER
Doubled MPLS over OC-Nc or 1/10GbE per site
LER
Wireless access
LER
. . .
LER
LER
LER
LER
Wireless access
LER
LER
Wireless access
Wireless access
Wireless access
MPLS routers at the customer switching
centers/content sites/Internet/PSTN access points
etc.
BGR Border Gateway Router (with firewalls) LER
Label Edge Router MGW Media (PSTNltgtVoIP)
Gateway
18
Implementation of Connectivity Requirements Using
OCS ITN Adaptive-Mesh
Wireless access
Wireless access
Content site / Data center SAN
Wireless access
. . .
LER
LER
. . .
PSTN
. . .
LER
LER
LER
. . .
IM
IM
LER/ MGW
7-site 2xSTS-N-rate Adaptive-Mesh over OC-4N
ring
IM
IM
IM
IM
Internet
IM
Doubled 5-site STS-N-rate Adaptive-Mesh over
OC-4N ring interconnects the four regional
Adaptive-Mesh networks and the PSTN/Internet
access points See next slide ?
LER/ BGR
IM
IM
IM
IM
PSTN
. . .
MGW
IM
IM
IM
Internet
IM
11 Protected, Link-aggregated MPLS over STS-Nc
PPP links
LER
LER
IM
IM
BGR
IM
7-site 2xSTS-N inter-site rate
Adaptive-Mesh over OC-4N ring
. . .
IM
IM
7-site 2xSTS-N inter-site rate
Adaptive-Mesh over OC-4N ring
Wireless access
IM
. . .
LER
IM
LER
IM
IM
IM
IM
IM
Doubled MPLS over OC-Nc or 1/10GbE per site
LER
LER
IM
IM
Wireless access
LER
. . .
LER
LER
LER
Wireless access
LER
LER
LER
Wireless access
Wireless access
Wireless access
MPLS routers at the customer switching
centers/content sites/Internet/PSTN access points
etc.
BGR Border Gateway Router (with firewalls) LER
Label Edge Router MGW Media (PSTNltgtVoIP)
Gateway
IM OCS Intelligent Transport Network IF module
19
Adaptive-Mesh for On-Demand E-Science Grids

• Summary
• Customer controllable, packet-layer transparent
  network service
• Packet transport using circuits of dynamically
  optimized bandwidth
• Maximized data throughput based on realtime
  traffic load patterns

Adaptive-Mesh Network Service User groups
private multi-service backbone -- just without
the need for the users to spend capital or
resources on deploying or operating their
backbone network