ICFA Standing Committee on Interregional Connectivity SCIC

1

• ICFA Standing Committee on
  Interregional Connectivity (SCIC)

Harvey B. Newman California Institute of
TechnologyICFA Meeting, BeijingAugust 20, 2004
2
Extra Slides Follow
3
ICFA Report Networks for HENPGeneral
Conclusions (3)

• We must Remove Firewall Bottlenecks Another
  Digital Divide also at some Major HEP Labs
• Firewall systems are so far behind the needs that
  they wont match the data flow of Grid
  applications. The maximum throughput measured
  across available products is limited to a few X
  100 Mbps !
• It is urgent to address this issue by designing
  new architectures that eliminate/alleviate the
  need for conventional firewalls. For example,
  Point-to-point provisioned high-speed circuits as
  proposed by emerging Light Path technologies
  could remove the bottleneck.
• With endpoint authentication as in Grid AAA
  systems, the point-to-point paths are private,
  intrusion resistant circuits, so they should be
  able to bypass site firewalls if the endpoints
  (sites) trust each other.

4
SCIC Recommendation 1Work on the Digital
Dividefrom Several Perspectives

• Work on Policies and/or Pricing pk, in, br, cn,
  SE Europe,
• Share Information Comparative Performance and BW
  Pricing
• Find Ways to work with vendors, NRENs, and/or
  Govts
• Exploit Model Cases e.g. Poland, Slovakia, Czech
  Republic
• Inter-Regional Projects
• South America CHEPREO (US-Brazil) EU ALICE
  Project
• GLORIAD, Russia-China-US Optical Ring
• Virtual SILK Highway Project (DESY) FSU
  satellite links
• Help with Modernizing the Infrastructure
• Design, Commissioning, Development
• Provide Tools for Effective Use Monitoring,
  Collaboration
• Participate in Standards Development Open Tools
• Advanced TCP stacks Grid systems
• Workshops and Tutorials/Training Sessions
• For Example Rio DD and HEPGrid Workshop,
  February 2004
• Raise General Awareness of the Problem
  Approaches to Solutions

5
On SCIC Recommendation 1 Work to Close the
Digital Divide Help Bring the Needed Networks
to All Regions

• ICFA Members should work vigorously towards this
  goal Locally, Nationally and Internationally
• Why ?
• Physicists from all world regions have the Right
  to be full partners
• It is the basis of our global community, and
  our largest projects
• Involvement of students, and outreach to the
  community is vital to our field. In modern
  times, this is founded on networks.
• How ? We are the prototypical global
  community
• Developments by HENP of Grids, state-of-the-art
  networks and systems for collaborative work on a
  worldwide scale represent a unique
  opportunity, for science and society
• Work with SCIC other cognizant organizations
• And If We Dont ?
• We fail as the first global community in
  science

6
Recommendation 2 Strongly Supportthe Monitoring
Work

• The IEPM Project http//www-iepm.slac.stanford.ed
  u/ Led by SLAC with help from FNAL and CERN
• This is Imperative, to
• Quantify and Bridge the Digital Divide
• Continue to Work with ICTP and Extend the
  Monitoring Coverageof Developing Countries
• Special Emphasis on Africa and Remote Regions
  Least 2 Hosts Monitored in Each Developing
  Country
• REQUEST ICFA (and Other) Assistance Find Sites
  and Contacts in
• Latin America Venezuela, Costa Rica, Honduras,
  El Salvador, Belize, Panama, Bolivia
• Africa Burkino Faso, Egypt, Ghana, Malawi,
  Nigeria, Senegal, Somalia, Kenya, Libya, Nigeria,
  Sudan
• Vietnam, Belarus
• Need Funding Agency Representatives are Asked to
  Help/Advise
• US DOE or NSF for IEPM at SLAC
• EU for ICTP that is working with IEPM for
  Developing Nations

7
Recommendations 3 4 Digital Divide
Workshops and World Map/Website

• 1st ICFA SCIC DD and HEPGrid Workshop
• February 16-20 2004 in Rio de Janeiro (near
  LISHEP)
• Then 1 Workshop Per Year, at Sites that Need
  Help
• Project to Build HENP World Network Map Updated
  and Maintained on a Web Site, Backed by Database
  
• Systematize and Track Needs and Status
• Share Information On
• Links Bandwidths Pricing Vendors
  Technologies
• Problems Overloading ( Where) Quality
  Peering, etc.
• Requirements Are They Being Met ?
• Identify Urgent Cases Focus on Opportunities to
  Help
• Funding Did Not Materialize in 2003 Continue
  to Seek Help (Manpower) and Funds

8
SCIC Work in 2004

• Continue Digital Divide Focus More In-Depth
  Information
• In Europe with TERENA
• In Asia with APAN and KEK
• In US, with Internet2 and ESnet
• On South America, with AMPATH, Internet2, RNP,
  et al.
• Continue on Africa, with Jensen and ICTP Trieste
  
• Set Up HENP Networks Web Site (Get Support
  and/or Funding)
• Continue and if Possible Strengthen Monitoring
  Work (IEPM)
• Continue Work on Specific Improvements, Case by
  Case
• Brazil and South America, with RNP
• Russia
• Pakistan (?) India (?) China (?)
• Romania
• Follow the World Summit on the Information
  Society Process
• Watch Requirements the Lambda Grid Grid
  Analysis Revolutions
• Encourage Creation of a New Culture of
  Collaboration

9
World Summit on the Information Society(WSIS)
Geneva 12/2003 and Tunis in 2005

• In 2001, the UN General Assembly adopted a
  resolution endorsing the organization of the
  World Summit on the Information Society (WSIS),
  under UN Secy General Kofi Annan, with the ITU
  taking the lead role in its preparation.
• GOAL Create An Information Society A Common
  Definition was adopted in the Tokyo
  Declaration, Jan. 2003 One in which highly
  developed ICT networks, equitable and ubiquitous
  access to information, appropriate content in
  accessible formats and effective communication
  can help people achieve their potential
• ICFA SCIC has been quite active in the WSIS in
  Geneva (12/2003)
• Second Phase (Tunis 2005) Preparation Process
  Underway

10
Role of Science in the Information Society WSIS
2003-2005

• HENP Active in WSIS
• CERN RSIS Event
• SIS Forum CERN/Caltech Online Stand at WSIS I
  (Geneva 12/03)
• Visitors at WSIS I
• Kofi Annan, UN Secy General
• John H. Marburger, Science Adviser to US
  President
• Ion Iliescu, President of Romania and Dan Nica,
  Minister of ICT
• Jean-Paul Hubert, Ambassador of Canada in
  Switzerland
• Planning Underway for WSIS II Tunis 2005

11
Role of Sciences in Information Society.
Palexpo, Geneva 2003

• Demos at the CERN/Caltech RSIS Online Stand
• Advanced network and Grid-enabled analysis
  demonstrations
• Monitoring very large scale Grid farms with
  MonALISA
• Distance diagnosis and surgery using Robots with
  haptic feedback (Geneva-Canada)
• World Scale multisite multi-protocol
  videoconference with VRVS (Europe-US-Asia-Sout
  h America)
• Music Grid live performance with bands at St.
  Johns, Canada and the Music Conservatory of
  Geneva on stage

12
ICTP 2nd Open Round Table on Developing
Countries Access to Scientific Information

• STATEMENT AFFORDABLE ACCESS TO THE INTERNET
• FOR RESEARCH AND LEARNING
• Scholars from across the world meeting at the
  Abdus Salam International Centre for Theoretical
  Physics (ICTP) in Trieste 10/2003 were
  concerned to learn of the barrier to education
  and research caused by the high cost of Internet
  access in many countries.
• The Internet enables the use of content which is
  vital for individuals and for institutions
  engaged in teaching, learning and research. In
  many countries use of the Internet is severely
  restricted by the high telecommunications cost,
  leading to inequality in realising the benefits
  of education and research. Research staff and
  students in countries with liberal
  telecommunications policies favouring educational
  use are gaining social and economic advantage
  over countries with restrictive, high-cost
  policies. The potential benefits of access to the
  Internet are not available to all.
• The signatories to this message invite scholars
  in every country to join them in expressing
  concern to governments and research funding
  agencies at the effect of high telecommunications
  costs upon individuals and institutions
  undertaking teaching, learning and research. The
  situation in many countries could be improved
  through educational discounts on normal
  telecommunications costs, or through the lifting
  of monopolies. It is for each country to
  determine its own telecommunications policies but
  the need for low-cost access to the Internet for
  educational purposes is a need which is common to
  the whole of humankind.

13
AARnet SXTransport Project in 2004

• Connect Major Australian Universities to 10 Gbps
  Backbone
• Two 10 Gbps Research Links to the US (X 64 in BW
  !)
• Aarnet/USLIC Collaboration on Net RD Starting
  Fall

14
Romania Inter-City Links of 2 to 6 Mbps in 2002
Improved to 155 or 34 Mbps in 2003-4
GEANT-Bucharest Link Improved 155 to 622 Mbps

RoEduNetAugust 2004
Planning Dark FiberInterCity Backbone
15
ICFA/SCIC Network Monitoring

• Prepared by Les Cottrell, SLAC, for
• ICFA
• www.slac.stanford.edu/grp/scs/net/talk03/icfa-aug0
  4.ppt

16
View from CERN

• Confirms view from N. America

From the PingER project August 2004.
17
From Developing Regions
Novosibirsk
Novosibirsk
NSK to Moscow used to be OK but loss went up in
Sep. 2003 GLORIAD may help
Brazil (Sao Paolo)
As expected Brazil to L. America is good Actually
dominated by Brazil to Brazil To Chile Uruguay
poor since goes via US
18
New features in works (with NIIT)

• Improve new site set-up tools
• Improve management
• Discover non working links faster
• Improve access to data and meta data
• Provide data base with lat/long, country etc.
• Add web services access
• Improve visualization
• Provide map with drill down to node information
• Automate production of long term trend plots for
  regions
• More node selection capabilities
• Traceroute measurement and analysis

19
History of Bandwidth Usage One large Network
One Large Research Site
ESnet Accepted Traffic 1/90 1/04Exponential
Growth Since 92Annual Rate Increased from 1.7
to 2.0X Per Year In the Last 5 Years
SLAC Traffic 300 Mbps ESnet LimitGrowth in
Steps 10X/4 YearsProjected 2 Terabits/s by
2014
AMS-IX Traffic Growth Similar 100 Per Year
20
Internet Growth in the World At Large
Amsterdam Internet Exchange Point Example
5 MinuteMax
20 G
Avg
10 G
Some Growth SpurtsTypically In Summer-Fall
The Rate of HENP Network Usage Growth (100 Per
Year) is Similar to the World at Large
21
SURFNet (NL) Traffic Growth 1999-2004
Annual Growth Factor 2.4 Times 10 PetaBytes/Year
22
APAN Concept 2004
100 Gbps
2.4Gbps
Europe
USA
10Gbps
35Gbps
Russia
Korea
12.5 Gbps(?)
2Gbps
6.2Gbps
10Gbps
Japan
China
Taiwan
622Mbps
2.5Gbps
2.55Gbps
12.5 Gbps(?)
155Mbps
Hong Kong
Singapore
622Mbps
23
China Japan Link
6TNet
Dragon Tap CNGI XP
BUPT
CSTNET
CERNET
KR
WIDE DIX-4
SINET
WIDE DIX-6
E over MPLS
2.5Gbps SONET
Conv.
APAN GbE
Genkai GbE
IPv6 GE
IPv4 GE
JGN
apan
LA
SL
2.5Gbps SONET (2.5Gbps x 2 in future)
GbE/SONET Converter
TransPAC
GbE/SONET Converter
JAPAN
China
10G?x 2 in 2004
24
APAN-Taiwan
622M 1G
6.2G (now) 2.5 to Seattle 3.7 to PAIX
622M
2.5Gbps
2.5G
622M
Hong Kong HARNet
NL-Amsterdam AMSIX
155M
Singapore SingAREN
25
New TransPAC links
Cisco ONS
Router
Cisco ONS15454 ML x 4 OC192IR x 1 XC-10G x 2
TransLIGHT
Carrier ADM
10GigE
Cisco ONS15454 ML x 4 OC192IR x 1 XC-10G x 2
SL Switch
Tokyo XP
CHIN
SONET STS12c x 16 Unprotected
SONET STS12c x 16
Abilene
Pro8812 OC192 POS
Lambda
LOSA
SONET STS192c
Tyco LA?
Carrier-LA
Tyco Shinagawa
SONET STS192c Protected

• NICT will get- 10Gbps single or low-priority
  SONET ring circuit from Carrier-LA to StarLight.
  - a single International circuit with the same
  SONET framing from Tokyo XP to Carrier-LA.
• IU will secure NSF grant extension of 9 months,
  for - 10Gbps high-priority SONET ring circuit
  (10G) from Carrier-LA to Abilene-LA.- the other
  single International circuit of the same SONET
  framing from Tokyo XP to Carrier-LA.
• IU will synchronize purchase of circuits with
  NICTsame carrier Aug. 2004 April 2005
  Contract

26
JGN2 Japan Gigabit Network

1. JGN2 Objectives and Vision Promoted through
links between industry, academia, government
and the regions Basic/fundamental RD,
applied RD and demonstration of
wide-ranging information and communications
technologies Stimulate regional RD activities
practical research activities Explore Prospects
of the Future IT society
2. JGN2 Features Open testbed network for
research and development Nationwide network
and access points in all prefectures (63 in
total) Maximum 20 Gbps backbone network 6
advanced Optical Cross Connects (OXC) for
high-speed data exchange Dark fiber
environment to enable the execution of various
experiments Japan-US line part of the network
(scheduled to go on-line in August 2004)
3. More Information Contact
jgn2center_at_jgn2.jp
27
CIREN

• CIREN Consortium of International Research and
  Education Networks
• Most of the Research Education Networks have
  joined this proposal, for expanding the global
  collaborations.
• Principal Investigator Michael McRobbie
  (Indiana Univ.)
• Co-PIs Doug Van Houweling (Internet2)
  Dai Davies (DANTE)
• Shigeki
  Goto (APAN)
• Nelson Simoes da Silva (CLARA)

28
CIREN Requirements Commitment

• Support of RE Community Major Networks have
  joined this proposal.
• Network Topology US Tokyo - HK
• 
  Busan SG (IN)
• Network RD NOC Collaborations, HOPI(Hybrid
  Optical and Packet Infrastructure), Measurement
  Infrastructure, E2E Performance, Security
  Center, etc.
• Support of NSFs CyberInfrastructure Global
  deployment of HENP, VLBI, NCMIR, TeraGrid, etc.
• Extension to other region Connection to India
  will be studied.

29
TransPAC/APAN Observatory Systems

• Two categories of services
• Advanced backbone data
• A program that supports the collection and
  dissemination of network data
• Backbone data collected by NOC using equipment
  located in TransPAC/APAN nodes. Data item and
  data format will become the same as those of
  Abilene observatory.
• http//abilene.internet2.edu/observatory/
• Just started by the Joint TransPAC and APAN-JP
  NOC members.
• http//www.jp.apan.net/NOC/Observatory/
• Collocation Systems assisted by NOC engineers
• Data collected by respective research projects
  using their own equipments collocated in XP.
• Accounts should be provided for NOC operators and
  engineers.
• Permission of exporting data should be authorized
  by the projects.

30
Observatory deployment into APAN

• That Observatory system should be widely deployed
  in APAN.

31
SuperSINET Updated Map Oct. 2003

SuperSINET 10 Gbps
Intl Circuit 5-10 Gbps

Domestic Circuit 30 100 Mbps

• SuperSINET
• 10 Gbps IP Tagged VPNs
• Additional 1 GbE Inter-University
  Wave For HEP
• 10 Gb to NY 10 GbE Peerings ESNet,
  Abilene and GEANT

32
CERNET Interconnections
33
(J.P. Wu, APAN 7/2004) Why is IPv6 needed?

• Much larger address space
• IPv6 Addresses 3.4X1038
• Trust network real IP address access
• Improved routing
• Route aggregation reduces the size of routing
  tables
• Simplified header reduces router processing loads
  
• Enhanced security and QoS
• Mandatory IPsec support all fully IPv6 compliant
  devices
• Improved support for mobile IP and mobile
  computing devices
• IP is everywhere
• Data, Voice, Audio and Video integration is a
  Reality
• Regional Registries apply a strict allocation
  control

34
DoD IPv6 Timeline
Successful Transition to IPv4 from NCP in
1982.12.31
35
IPv6

• Korea to invest 160 million in IPv6 by 2007
• The Korean government aims to put the nation in a
  leadership role in the world-wide Internet
  equipment market and make it an internet super
  power by commercialising IPv6 technology early
  on.
• (Source Korea Herald, 19 Sep 2003).
• NTT has a worldwide full dual stack backbone and
  more than three years experience to provide
  commercial IPv6 connectivity services.

36
APAN - General conclusion

• APAN has completed a very successful
  establishment phase
• Now it must move forward as an active mature
  network, supporting the broad, diverse interests
  of e-research in Asia, with all the
  organizational and political complexities that
  entails
• Old issues continue
• Connections to US remain strong, but a true
  Asia-Pacific backbone (like GEANT) has yet to be
  built
• more north/south linkages within Asia are needed
  and better connectivity to Europe, possibly via
  west Asia
• Applications are developing but stronger
  intra-Asia applications, grid applications in
  particular, need support
• APAN Working Groups play a vital role here

37
Updates on Research Networks(Korea and APAN)

• Dongchul Son
• Center for High Energy Physics
• Kyungpook National University
• Daegu, 702-701, Korea
• son_at_knu.ac.kr
• August 2004, for ICFA-SCIC

38
Research Networks in Korea

• Two backbone networks for advanced networks
  research/applications in Korea
• KREONET (Korea Research Environment Open NETwork)
• Backbone of 10 G in the Daejeon Area (called
  SuperSIREN) and 2.5 G from Daejeon to 10 Regional
  Centers (Including major HEP sites in Daegu,
  Seoul, etc.) User Sites are connected with 1 G
• KOREN (KOrea Advanced Research Network)
• 10 G between Seoul and Daejeon and 2.5 G in Ring
  configuration of major four cities (Daejeon
  Daegu Busan Gwangju Daejeon)
• Abroad
• APII (Asia-Pacific Information Infrastructure) to
  Japan, China, North America, Malaysia, Singapore
• TEIN (Trans-Eurasia Information Network) to France

39
Research Networks in Korea

• Updates on Interregional connectivity
• APII upgrade Korea to US via CAnet 4 is now
  1.24 Gbps (2622 Mbps)
• KREONet2 and KOREN, the national research
  networks of Korea, now transit traffic from the
  Pacific Northwest GigaPoP (PNWGP) in Seattle,
  where Korea's transpacific link lands, to
  StarLight in Chicago via Canada's optical
  research network CAnet 4. KREONet2 and KOREN,
  two distinct research communities sharing one
  link, disconnected its 155Mbps connection to STAR
  TAP, on April 30.
• The networks now peer with a global community of
  research and education networks at StarLight,
  that supports 1Gbps, 2.5Gbps and 10Gbps links.
• APII-Japan and KOREN to US/Europe via TransPAC 1
  Gbps
• KOREN from/to Kyushu, Japan has two 1Gbps
  connections, called Genkai/Hyeonhai network.
• The network now peer via TransPAC with
  US/European networks with a 1 Gbps link since
  June 2004
• APII-China A 155 Mbps link opened and another
  155 Mbps expected soon.
• A 155 Mbps link between KOREN/CERNET was opened
  July 30, 2004 and another 155 Mbps link between
  KREONET/CERNET is expected soon, totaling 310
  Mbps.
• TEIN upgrade PCR 34 Mbps/SCR 17 Mbps
• TEIN between RENATER, France and KOREN, Korea was
  upgraded to PCR 34 Mbps/SCR 17 Mbps on November
  3, 2003.
• It is expected to support 155 Mbps in 2004.

40
Network test results

• Inside Korea
• 925 Mbps /1G achieved and tests of 2.5G are
  under way (KAIST-KNU)
• Korea(KNU) ?? US (FNAL, Caltech)
• 916 Mbps/1G achieved (22/06/04)
• 2 paths exist (1.2G, 1G)

NEW
41
KR-US/CA Transpacific connection

• Participation in Global-scale Lambda Networking
• Two STM-4 circuits (1.2G) KR-CA-US
• Global lambda networking North America,
  Europe,
• 
  Asia Pacific, etc.

Global Lambda Networking
KREONET/SuperSIReN
CANet4
StarLight
STM-4 2
Chicago
APII-testbed/KREONet2
PacWave
Seattle
42
New Record!!! 916 Mbps from CHEP to Caltech
(22/06/04)
Subject UDP test on
KOREN-TransPAC-Caltech Date
Tue, 22 Jun 2004 134725 0900 From
"Kihwan Kwon"
To

root
_at_sul Iperf ./iperf -c socrates.cacr.caltech.edu
-u -b 1000m -------------------------------------
----------------------- Client connecting to
socrates.cacr.caltech.edu, UDP port 5001 Sending
1470 byte datagrams UDP buffer size 64.0 KByte
(default) ----------------------------------------
-------------------- 5 local 155.230.20.20
port 33036 connected with 131.215.144.227 port
5001 ID Interval Transfer
Bandwidth 5 0.0-2595.2 sec 277 GBytes
916 Mbits/sec

USA TransPAC
G/H-Japan
KNU/Korea
Max. 947.3Mbps
43

Study into European Research and Education
Networking as Targeted by eEurope
www.serenate.org
SERENATE is the name of a series of strategic
studies into the future of research and education
networking in Europe, addressing the local
(campus networks), national (national research
education networks), European and
intercontinental levels. The SERENATE studies
bring together the research and education
networks of Europe, national governments and
funding bodies, the European Commission,
traditional and "alternative" network operators,
equipment manufacturers, and the scientific and
education community as the users of networks and
services.
From Summary and Conclusions by D.O. Williams,
CERN ?
44
Dai Davies SERENATE Workshop Feb. 2003
www.serenate.org
Ratio to 114 If Include Turkey, MaltaCorrelated
with the Number of Competing Vendors
45
Data illustrating the digital divide
(D.O.Williams, LISHEP2004)
46
GÉANT January 2003 (D.O.Williams, LISHEP2004)
47
High speed University Connections (D.O.Williams,
LISHEP2004)
48
Optics and FibresMessage to NRENs or Natl
Initiatives

• If there is one single technical lesson from
  SERENATE it is that transmission is moving from
  the electrical domain to optical.
• The more you look at underlying costs the more
  you see the need for users to get access to
  fibre.
• When theres good competition users can still
  lease traditional communications services
  (bandwidth) on an annual basis.
• But Without enough competition prices go through
  the roof.
• A significant divide exists inside Europe
  with the worst countries Macedonia, B-H,
  Albania, etc. 1000s of times worse off than the
  best. Also many of the 10 new EU members are 5X
  worse off than the 15 present members.
• Our best advice has to be if youre in a mess,
  you must get access to fibre.
• Also try to lobby politicians to introduce real
  competition
• In Serbia still a full telecoms monopoly
  the two ministers talked and the research
  community was given a fibre pair all around
  Serbia !

49
Virtual Silk Highway
The SILK Highway Countriesin Central Asia
the Caucasus

• Hub Earth Station at DESY with access to the
  European NRENs and the Internet via GEANT
• Providing International Internet access directly
• National Earth Station at each Partner site
• Operated by DESY, providing international access
• Individual uplinks, common down-link, using DVB
• Currently 4 Mbps Up 12 Down for 350k/Year

Note Satellite Links are a Boon to the Region,
but Unit Costs are Very High compared to
Fiber.There is a Continued Need for Fiber
Infrastructure
50
VRVS on Windows
KEK (JP)
VRVS (Version 3) Meeting in 8 Time Zones
Caltech (US)
RAL (UK)
Brazil
CERN (CH)
AMPATH (US)
Pakistan
SLAC (US)
Canada
36k hosts worldwide Users in 106 Countries 2-3X
Growth/Year
AMPATH (US)
51
UltraLight Developing Advanced Network Services
for Data Intensive HEP Applications

• UltraLight a next-generation hybrid packet- and
  circuit-switched network infrastructure
• Packet switched cost effective solution
  requires ultrascale protocols to share 10G ?
  efficiently and fairly
• Circuit-switched Scheduled or sudden overflow
  demands handled by provisioning additional
  wavelengths Use path diversity, e.g. across the
  US, Atlantic, Canada,
• Extend and augment existing grid computing
  infrastructures (currently focused on
  CPU/storage) to include the network as an
  integral component
• Using MonALISA to monitor and manage global
  systems
• Partners Caltech, UF, FIU, UMich, SLAC, FNAL,
  MIT/Haystack CERN, NLR, CENIC, Internet2
  Translight, UKLight, Netherlight UvA, UCL, KEK,
  Taiwan
• Strong support from Cisco and Level(3)

52
SC2004 HEP Network LayoutPreview of Future Grid
systems
Brazil
UK
Japan
SLAC
Australia
FNAL
StarLight
210 GbpsNLR
LA
2 Metro 10 Gbps Waves LA-Caltech

• Joint Caltech, CERN, SLAC, FNAL, UKlight, HP,
  Cisco Demo
• 6 to 8 10 Gbps waves to HEP setup on the show
  floor
• Bandwidth challenge aggregate throughput goal
  of 40 to 60 Gbps

Caltech CACR
CERN Geneva
53
GLIF Global Lambda Integrated Facility
GLIF is a World Scale Lambda based Lab for
Application and Middleware development, where
Grid applications ride on dynamically configured
networks based on optical wavelengths ...
Coexisting with more traditional packet-switched
network traffic 4th GLIF WorkshopNottingham UK
Sept. 2004
10 Gbps Wavelengths For RE Network Development
Are Prolifering, Across Continents and Oceans
54
18 State Dark Fiber InitiativesIn the U.S. (As
of 2/04) California (CALREN), Colorado
(FRGP/BRAN)Connecticut Educ. Network,Florida
Lambda Rail, Indiana (I-LIGHT), Illinois
(I-WIRE), Md./DC/No. Virginia (MAX),Michigan,
Minnesota, NY New England (NEREN), N.
Carolina (NC LambdaRail), Ohio (Third Frontier
Net) Oregon, Rhode Island (OSHEAN), SURA
Crossroads (SE U.S.), Texas,Utah, Wisconsin
The Move to Dark Fiber is Spreading
FiberCO
55
PROGRESS Rays of Hope in SE Europe (Sk, Pl, Cz,
Hu, )
1660 km of Dark Fiber CWDM Links, up to 112 km.
1 to 4 Gbps (GbE) August 2002 First NREN in
Europe to establish Intl GbE Dark Fiber Link, to
AustriaApril 2003 to Czech Republic. Planning
10 Gbps Backbone dark fiber link to Poland this
year.
56
2 Years Ago 4 Mbps was the highest bandwidth
link in Slovakia
57
The Advantage of Dark Fiber CESNET Case Study
(Czech Republic)
2513 km Leased Fibers (Since 1999)
Case Study ResultWavelength ServiceVs. Fiber
Lease Cost Savings of 50-70 Over 4 Yearsfor
Long 2.5G or 10G Links
58
SURFNet6 in the Netherlands 3000 km of Owned
Dark Fiber
40M Euro ProjectScheduled Start
Mid-2005Support Hybrid Grids
59
Dark Fiber in Eastern Europe Poland PIONIER
Network
2650 km Fiber Connecting16 MANs 5200 km and
21 MANs by 2005

• Support
• Computational GridsDomain-Specific Grids
• Digital Libraries
• Interactive TV
• Addl Fibers for e-Regional Initiatives

60
Asia-Pacific Advanced Network (APAN)

• Objectives are
• to coordinate and promote network technology
  developments and advances in network-based
  applications and services
• to coordinate the development of an advanced
  networking environment for research and education
  communities in the Asia-Pacific region and
• to encourage and promote global cooperation to
  help achieve the above.
• http//www.apan.net/

61
DOE UltraScience NetA Lambda-Switching Testbed

• Enough lambdas (2 initial) to make switching real
• Explore Light-Paths for high-end transport
• Connect four hubs close to large DOE science
  users (but let the user labs pay last-mile costs)
• Hubs in Sunnyvale, Seattle, Chicago, Atlanta
• Provide an evolving matrix of switching
  capabilities
• Separately fund research projects (e.g.,
  high-performance protocols, control,
  visualization) that will exercise the network and
  directly support applications at the host
  institutions

62
UltraScience Net Physical View From 50,000 feet
ORNL Connector
Bill Wing, ORNLJuly 2004
63
UltraScience Control-Plane Phase I
Core Director
Seattle
Core Director
Chicago
host
VPN
Sunnyvale
host
VPN
VPN
Core Director
IP network
host
ORNL
VPN
Core Director
host
lambda
VPN
TL1
64
UltraScience Net Summary

• Atlanta-Chicago link to be up this summer
• Connecting ORNL to it will initially be via
  Atlanta
• Chicago-Sunnyvale paced by NLR
• Initial 10-Gig test circuit, OC192 SONET follows
• PNNL fiber schedule will pace their connection
• Expect E-to-E tests by fall
• Expect user traffic before SC2004

65
LambdaStation On UltraScience Net

• A Joint Fermilab and Caltech project
• Based on the UltraLight concept
• Enabling HEP applications to send high throughput
  traffic between mass storage systems across
  advanced network paths
• Dynamic Path Provisioning across Ultranet, NLR
  Plus an Abilene standard path

66

?
?
67
Networks, Grids and HENP

• Network backbones and major links used by HENP
  experiments are advancing rapidly
• To the 2.5-10G range in 2 years much faster
  than Moores Law
• HENP is learning to use long distance 10 Gbps
  networks effectively
• 2003 Developments to 7.5 Gbps flows over 16,000
  km
• Transition to a community-owned or leased fibers
  for RE has begun in some areas us, ca, nl,
  pl, cz, sk or is considered de, ro IEEAF
• End-to-end Capability is Needed, to Reach the
  Physics Groups
• Removing Regional, Last Mile, Local Bottlenecks
  and Compromises in Network Quality are now
  On the critical path, in all world regions
• Digital Divide Network improvements are
  especially neededin SE Europe, Latin America,
  China, Russia, Much of Asia, Africa
• Work in Concert with Internet2, Terena, APAN,
  AMPATH, GLORIAD TransLight, GLIF,
  UltraLight, the Grid projects and the
  Global Grid Forum