SPRACE KyaTera UltraLight Proposal

1
SPRACE KyaTera / UltraLight Proposal

• VI D0SAR Workshop
• São Paulo, Brazil
• September 16, 2005

Rogério L. Iope Universidade de Sao Paulo (Grad.
Research Assistant for SPRACE)
2
e-Science Data Gathering, Analysis, Simulation,
Collaboration

• Scientific discoveries increasingly driven by
  data collection
• Computationally intensive analyses
• Massive data collections
• Data distributed across networks of varying
  capability
• Internationally distributed collaborations
• New approaches to enquiry based on
• Deep analysis of huge quantities of data
• Interdisciplinary collaboration
• Large-scale simulation
• Smart instrumentation

• e-Science methods no longer optional but now
  vital to scientific competitiveness

3
e-Science Driving Global Cyberinfrastructure
CMS
TOTEM
ALICE HI
ATLAS
LHCb B-physics

• e-Science is about providing significantly
  enhanced research infrastructure by utilizing
  distributed resources such as computers, storage
  devices, scientific instruments, and experts
  using information technology

4
e-Science Driving Global Cyberinfrastructure

• The enormous speedups of computers and networks
  have enabled simulations of far more complex
  systems and phenomena, as well as visualizing the
  results from many perspectives
• Advanced computing no longer restricted to a few
  research groups in a few fields, but pervades
  scientific and engineering research
• New data-intensive applications are driving
  seemingly insatiable demand for more bandwidth
• Groups collaborate across institutions and time
  zones, sharing data, complementary expertise,
  ideas, and access to special facilities without
  traveling
• Optical Networks are key to this vision
• Massive scalable bandwidth
• Protocol and bit-rate independence
• The ability to launch and scale new services on
  demand

• Photonic Networking the way to cope with IP
  traffic explosion

5
Overview of the UltraLight Project

• UltraLight is
• A collaboration of experimental physicists,
  computer scientists, and network engineers from
  BNL, Caltech, CERN, UF, FIU, FNAL, Internet2, UM,
  MIT, SLAC...
• to provide the network advances required to
  enable petabyte-scale analysis of globally
  distributed data
• An application-driven network RD program to
  explore the integration of cutting-edge network
  technology with the Grid computing and data
  infrastructure of HEP/Astronomy
• A non-standard core network with dynamic and
  varying bandwidth interconnecting globally
  distributed nodes
• An NSF-funded 4 year program to deliver a new,
  high-performance, network-integrated
  infrastructure
• Two primary, synergistic activities (source S.
  McKee)
• Network Backbone Perform RD / engineering
• Application Driver System Services RD /
  engineering

6
Overview of the UltraLight Project

• Main goals
• Engineer and operate a trans- and
  intercontinental optical network testbed
• Promote the network as an actively managed
  component
• Develop and deploy prototype global services
  which broaden existing Grid computing systems
• Enable physics analysis and discoveries by
  integrating and testing UltraLight in Grid-based
  physics production and analysis systems currently
  under development in ATLAS and CMS
• A three-phased plan
• Phase 1 (12 months) Implementation of network,
  equipment and initial services
• Phase 2 (18 Months) Integration and footprint
  expansion
• Phase 3 (18 Months) Transition to production
  (LHC physics eVLBI astronomy)

7
Overview of the UltraLight Project

• Project Management Team
• PI Harvey Newman (Caltech)
• Project Coordinator Rick Cavanaugh (UF)
• Network Coordinator Shawn McKee (UM)
• Applications Coordinator Frank van Lingen
  (Caltech)
• EducationOutreach Coordinator Laird Kramer
  (FIU)
• Physics Analysis User Community Coordinator
  Dimitri Bourilkov (UF)
• Wan-In-Lab Steven Low (Caltech)
• Project Coordination activities
• Regularly scheduled phone and video meetings
• Periodic face-to-face focus workshops
  (semi-annually or quarterly)
• Persistent VRVS room for collaboration
• Mail-lists
• Web-page portal (first prototype)

8
Overview of the UltraLight Project

• Some important UltraLight RD goals
• Basic Network Services
• Data transport protocols
• MPLS/QoS Services and Planning
• Optical Path Management Plans
• Optical Testbed
• Optical Exchange Point
• Network Monitoring
• Network Management and AAA
• Disk-to-disk data transfers
• Wan-In-Lab / DISUN
• HEP Application Services

9
(No Transcript)
10
Connectivity Diagram for UltraLight

• Source http//ultralight.caltech.edu/

11
The KyaTera Project

• A cooperative program proposed by FAPESP, as part
  of the TIDIA Program
• Main goal
• The establishment of an optical fiber network
  infrastructure connecting laboratories for
  research, development and demonstrations of
  technologies for advanced Internet applications
• Network infrastructure based upon the concept of
  dark fibers reaching directly to the research
  laboratories (FTTLab)
• The name KyaTera comes from
• Kya (net in Tupi-Guarani)
• Tera (greek teras monster)

12
The KyaTera Project

• Composed by a dark fiber mesh spread over several
  cities among the State of São Paulo
• A large, geographically distributed laboratory
  facility for experimental tests of new network
  concepts and optical devices, new network
  protocols and services
• A platform for developing and deploying new high
  performance e-Science applications
• A stable, high performance network always
  co-exists with the experimental network
• new developments in the last do not interfere
  with the operation of the first

13
The KyaTera Project

• Research subjects for KyaTera organized in 3
  layers
• Physical Layer
• optical communications, new developments on fiber
  infrastructure
• Transport Layer
• protocols, interface standards, maanagement,
  monitoring, interoperability, etc, in optical
  networks
• Applications Layer
• automation and computer control of scientific
  instruments, Grid applications, HDTV, etc

14
(No Transcript)
15
WDM Fundamentals

• Wavelength-Division Multiplexing WDM
• An approach that can exploit the huge bandwidth
  available on fiber optic links
• Can manyfold the capacity of existing networks by
  transmitting many channels simultaneously on a
  single fiber optic line
• The optical transmition spectrum is carved up
  into a number of non-overlapping wavelength (or
  frequency) bands
• Multiple WDM channels from different end-users
  may be multiplexed on the same fiber
• Each wavelength supports a single communication
  channel operating at peak electronic speed
• By allowing multiple WDM channels to coexist on a
  single fiber, one can tap into the huge fiber
  bandwidth

• A more cost-effective alternative compared to
  laying more fibers

16
WDM - Parallelism on Optical Networking
Parallel lambdas will drive this decade the way
parallel processors drove the 1990s !
17
WDM Fundamentals

• WDM building blocks
• Light sources (laser diodes) and detectors
  (photodetectors, filters)
• Optical fibers (single-mode, multi-mode)
• Multiplexers and Demultiplexers
• Optical Add/Drop Multiplexers
• Optical amplifiers (e.g. EDFA)
• Photonic cross-connect switches
• Transponders

18
WDM Fundamentals

• A wavelength-routed optical WDM network consists
  of a photonic switching fabric comprising active
  optical switches connected by fiber links forming
  any arbitrary physical topology
• Each node equipped with a set of transmitters and
  receivers (which may be wavelength tunable)
• The basic mechanism of communication in such a
  network is a lightpath
• Lightpath an all-optical communication channel
  (a path) between 2 nodes (it can span more than
  one fiber link!)
• The intermediate nodes in this fiber path route
  the lightpath in the optical domain using their
  active optical (photonic) switches
• The end-nodes of the lightpath access the signal
  with transmitters and receivers that are tuned
  to the wavelength on which the lightpath operates

19
WDM Fundamentals

• Photonic switches protocols like GMPLS are key
  elements to address new goals, and implement a
  multi-tiered and scalable IP/Optical network

20
WDM Fundamentals

• In wavelength-routed WDM networks, a control
  mechanism is needed to set up and take down the
  optical connections (lightpaths)
• A successful data transfer event between 2 nodes
  has three phases
• Connection establishment
• Data transfer
• Connection release
• During first phase, a few control signaling
  packets are exchanged between network resources,
  aiming to establish a lightpath with an assigned
  wavelength
• If it succeeds, a lightpath is established, and
  data transfer occur through this circuit from
  source to destination
• When the transfer is completed, control packets
  are again exchanged between the nodes, and the
  resources are released and made ready to be
  assigned for another connection

21
WDM Fundamentals

• A challenging networking problem is that, given a
  set of lightpaths that need to be established on
  the network, and given a constraint on the number
  of wavelengths,
• determine the routes over which these lightpaths
  should be set up
• determine the wavelengths that should be assigned
  to them so that the maximum number of lightpaths
  may be established
• If any switching/router node is also equipped
  with a wavelength-converter facility, then
  lightpaths can be established using diferent
  wavelengths on their routes from origin to
  destination
• This problem is referred to as the RWA problem

22
WDM Systems General layout
Transmissor
DWDM
Transponder DWDM
MUX DWDM
DEMUX DWDM
?1
fiber
fiber
Core Router
Core Router
?1
OXC
EDFA
EDFA
GBIC n
GBIC 1
GBIC n
GBIC 1
?n
?n
Transponder CWDM
Transponder CWDM
?cn
?c1
?cn
?c1
MUX CWDM
DEMUX CWDM
MUX CWDM
DEMUX CWDM
Border Router
Border Router
OADM 1
OADM 1
Border Router
Border Router
?c1
?c1
CWDM
CWDM
?cn
?cn
OADM n
OADM n
(Source M. Stanton - GIGA Project)
23
WDM Systems R-OADM Conceptual Architecture
Software Controlled Selectors (Pass-through/Add/Bl
ock)
Pass
Splitter
West
Pass-Through Wavelengths
DWDM Signal
Add
Pass
Software Controlled DEMUX
Add
block
block
drop
Add Wavelengths
Drop Wavelengths
Transponder Module
Network Element
Network Element
Network Element
Network Element
?3
Transponder Module
Drop Wavelengths
Add Wavelengths
Software Controlled DEMUX
drop
block
block
drop
Add
DWDM Signal
Pass
Pass-Through Wavelengths
East
Add
Splitter
Pass
Software Controlled Selectors (Pass-through/Add/Bl
ock)
24
The KyaTera testbed Reference Architecture
Ethernet Aggregation Switch
IP Router 10 GbE lt-gt l
MUX/DEMUX R-OADM
MUX/DEMUX R-OADM
Photonic Switch
Ethernet Aggregation Switch
Ethernet Aggregation Switch
MUX/DEMUX R-OADM
MUX/DEMUX R-OADM
MUX/DEMUX R-OADM
MUX/DEMUX R-OADM
IP Router 10 GbE lt-gt l
IP Router 10 GbE lt-gt l
Photonic Switch
Photonic Switch
25
The KyaTera testbed (example of a proposed
solution)
26
Enabling e-Science The KyaTera / UltraLight
Proposal

• Network support a critical aspect of
  Grid-enabled environments
• Commodity Internet is based on a best-effort
  delivery model, a vehicle excessively slow and
  unreliable for the huge masses of data being
  generated in emerging e-Science applications
• Deployment of Grids on wide-area scales is being
  severely restricted

27
Enabling e-Science The KyaTera / UltraLight
Proposal

• Optical networing a promising solution to these
  limitations
• Emerging lightpaths technologies are becoming
  more and more popular in the Grid community
• They can include the network resources as an
  integral Grid component, controlled by Grid
  schedulers in the same way as computing elements
  and storage resources
• The challenge
• A new management technology is needed to allow
  end-users to acquire network resources on demand,
  control end-to-end interconnections between peers
  (lightpaths), and share unused bandwidth in a
  flexible and collaborative way

28
The KyaTera / UltraLight Proposal

• Our project proposal
• To work on the problem of monitoring, managing,
  and optimizing the use of the networking
  resources present in next-generation
  user-controlled optical networks in real time
• To work in close partnership with the UltraLight
  Project and the KyaTera Project
• To use the optical networking infrastructure that
  is being made available by the KyaTera Project
• The KyaTera network insfrastructure, enhanced by
  an intelligent optical control plane middleware,
  will provide the basement for the deployment of
  the Grid-enabled Analysis Environment Service
  Architecture (GAE), a project being developed at
  Caltech and University of Florida, coordinated by
  Prof. Harvey Newman

29
The KyaTera / UltraLight Proposal

• Research will be done on provisioning end-to-end
  survivable optical connections in the testbed, as
  in a Grid environment, with an innovative use of
  the GMPLS control plane
• (this will be accomplished in a close partnership
  with the OptiNet lab experts)

(Drawing and text courtesy of Gustavo Pavani
OptiNet / UNICAMP)
30
Project Planning Milestones and Timeframe

• Milestones
• Provisioning of end-to-end optical connections
  between pairs of nodes
• Provisioning multilayer protocols and intelligent
  monitoring software agents, and research on RWA
  algorithms
• Deployment of routing/switching and control
  protocols to locate suitable lightpaths and
  schedule the networking resources
• Deployment of Grid Analysis Environment
• Job submissions and data transfers between sites
  over the distributed computing infrastructure
  looking for failures, malfunctioning and
  bottlenecks

31
Project Planning Milestones and Timeframe
I
II
III
IV
V
32
(No Transcript)