Title: ICFA Standing Committee on Interregional Connectivity SCIC
1- ICFA Standing Committee on
Interregional Connectivity (SCIC)
Harvey B. Newman California
Institute of TechnologyICFA Meeting,
TsukubaFebruary 13, 2003
2 SCIC in 2003
- Formed WGs in March Overall 9 Meetings in 12
Months - Strong Focus on the Digital Divide
- Presentations at Meetings and Workshops(e.g.
LISHEP, APAN, AMPATH, ICFA Seminar) - HENP more visible to governments in the WSIS
Process - Five Reports Presented to ICFA Today
- Main Report Networking for HENP H. Newman et
al. - Monitoring Working Group L. Cottrell
- Advanced Technologies WG R. Hughes-Jones,
O. Martin et al. - Digital Divide A. Santoro et al.
- Digital Divide in Russia V. Ilyin
3 SCIC in 2003A Period of Intensive Activity
- New Web Page http//cern.ch/ICFA-SCIC/
- Monitoring Les Cottrell (http//www.slac.stanfor
d.edu/xorg/icfa/scic-netmon) With Richard
Hughes-Jones (Manchester), Sergio Novaes (Sao
Paolo) Sergei Berezhnev (RUHEP), Fukuko Yuasa
(KEK), Daniel Davids (CERN), Sylvain Ravot
(Caltech), Shawn McKee (Michigan) - Advanced Technologies Richard Hughes-Jones,With
Vladimir Korenkov (JINR, Dubna), Olivier
Martin(CERN), Harvey Newman - The Digital Divide Alberto Santoro (Rio, Brazil)
- With V. Ilyin (MSU), Y. Karita(KEK), D.O.
Williams (CERN) - Also D. Son (Korea), H. Hoorani, S. Zaidi
(Pakistan), S. Banerjee (India), V. White
(FNAL), J. Ibarra and Heidi Alvarez (AMPATH) - Key Requirements Harvey Newman and Charlie Young
(SLAC)
4SCIC General Conclusion for 2002Setting the Tone
for 2003
- The scale and capability of networks, their
pervasiveness and range of applications in
everyday life, and HENPs dependence on networks
for its research are all increasing rapidly. - One recent development accelerating this trend is
the worldwide development and deployment of
data-intensive Grids - Especially as physicists develop ways to do data
analysis, and collaborate in a Grid-enabled
working environment - However, as the pace of network advances
continues to accelerate, the gap between the
economically favored regions and the rest of
the world is in danger of widening. - We must therefore work to Close the Digital
Divide - To make Physicists from All World Regions Full
Partners in the scientific process of Search
and Discovery - This is essential for the health of our global
experimental collaborations, our plans for
future projects, and our field.
5 2001 Transatlantic Net WG Bandwidth
Requirements
See http//gate.hep.anl.gov/lprice/TAN. The
2001LHC requirements outlook now looks Very
Conservative in 2003
6History One large Research Site
Much of the TrafficSLAC ? IN2P3/RAL/INFNvia
ESnetFranceAbileneCERN
Current Traffic 400 MbpsESNet
LimitationProjections 0.5 to 24 Tbps by 2012
7Rapid Network Advances and the Digital Divide
- The current generation of 2.5-10 Gbps network
backbones arrived in the last 15 Months in the
US, Europe and Japan - Major transoceanic links also are reaching 2.5 -
10 Gbps - Capability Increased 4 Times, i.e. 2-3 Times
Moores Law - This is a direct result of the continued
precipitous fall of network prices for 2.5 or 10
Gbps in these regions - There are strong prospects for further advances
that will cause the Divide to become a Chasm,
Unless We Act For the Rich Regions - DWDM More 10G wavelengths and/or 40G speeds on
a fiber - Advances in protocols, including TCP, to use
networks efficiently at 1-10 Gigabit/sec (and
eventually higher) speeds - Cheap Gigabit Ethernet (GigE) 10GigE in campus
metro backbones - Owned or leased wavelengths in the last mile,
the region, and across the country
8The Continued Rapid Fall of Network Prices
- Linked to the bad state of the world economy
- IT spending stagnated or declined, in the major
markets - Supply outstripped demand
- Telecom sector near a state of collapse
- Vendors were and are desperately seeking
revenues - Ironically the same market forces keep prices
high where there is strong pent-up demand
and/or little competition namely in the poorer
markets. - E.g. Russia, Southeast Europe, India, China,
Pakistan - Vendors jealous of their revenues
- Will not install new lower price technologies
- Governments do not help
9SuperSINET Updated Map October 2002
10(No Transcript)
11 2003 OC192 and OC48 Links Coming Into
ServiceNeed to Consider Links to US HENP Labs
12National Light Rail Footprint
- NLR
- Buildout Started November 2002
- Initially 4 10 Gb Wavelengths
- To 40 10Gb Waves in Future
US Transition beginning now to optical,
multi-wavelength RE networks
13HENP is Learning How to Use Gbps Networks Fully
Factor of 100 Gain in Maximum Sustainable
Thruput in 15 Months, On Some USTransAtlantic
Routes
- 9/01 105 Mbps 30 Streams SLAC-IN2P3 102
Mbps 1 Stream CIT-CERN - 11/5/01 125 Mbps in One Stream (modified
kernel) CIT-CERN - 1/09/02 190 Mbps for One stream shared on 2
155 Mbps links - 3/11/02 120 Mbps Disk-to-Disk with One Stream
on 155 Mbps link (Chicago-CERN) - 5/20/02 450-600 Mbps SLAC-Manchester on OC12
with 100 Streams - 6/1/02 290 Mbps Chicago-CERN One Stream on
OC12 (mod. Kernel) - 9/02 850, 1350, 1900 Mbps Chicago-CERN
1,2,3 GbE Streams, 2.5G Link - 11-12/02 FAST TCP 940 Mbps in 1 Stream
California-CERN
9.4 Gbps in 10 Flows California-Chicago
14HENP Major Links Bandwidth Roadmap (Scenario)
in Gbps
Continuing the Trend 1000 Times Bandwidth
Growth Per DecadeWe are Rapidly Learning to Use
Multi-Gbps Networks Dynamically
15HENP Lambda GridsFibers for Physics
- Problem Extract Small Data Subsets of 1 to 100
Terabytes from 1 to 1000 Petabyte Data Stores - Survivability of the HENP Global Grid System,
with hundreds of such transactions per day
(circa 2007)requires that each transaction be
completed in a relatively short time. - Example Take 800 secs to complete the
transaction. Then - Transaction Size (TB) Net
Throughput (Gbps) - 1
10 - 10
100 - 100
1000 (Capacity of
Fiber
Today) - Summary Providing Switching of 10 Gbps
wavelengthswithin 3-5 years and Terabit
Switching within 5-8 yearswould enable
Petascale Grids with Terabyte transactions,as
required to fully realize the discovery potential
of major HENP programs, as well as other
data-intensive fields.
16 History - Throughput Quality
Improvements from US
80 annual improvement Factor 100/8 yr
Bandwidth of TCP lt MSS/(RTTSqrt(Loss)) (1)
Progress but Digital Divide is Maintained
(1) Matthis et al., Computer Communication Review
27(3), July 1997
17NREN Core Network Size (Mbps-km)http//www.teren
a.nl/compendium/2002
100M
Logarithmic Scale
Leading
Nl
10M
Fi
Cz
Advanced
Hu
Es
1M
Ch
In Transition
It
Pl
Gr
100k
Ir
Lagging
10k
Ro
1k
Ukr
100
18Romania 155 Mbps to GEANT and BucharestInter-Ci
ty Links of 2, 8 and 34 Mbps in 2003
2 Mbps-POP
2 Mbps(backup)
8 Mbps
Botosani
Satu Mare
Baia Mare
Suceava
34 Mbps
GEANT connection
Iasi
155 Mbps
Bistrita
Zalau
Oradea
Piatra Neamt
Tg-Mures
Cluj
Arad
Bacau
Vaslui
Mircurea Ciuc
Timisoara
Alba Iulia
Focsani
Hunedoara
Sf. Gheorghe
Galati
Sibiu
Brasov
Buzau
Tîrgu Jiu
Rm.Vîlcae
Ploiesti
Pitesti
Tulcea
Resita
Braila
Tîrgoviste
Slobozia
Slatina
RoEduNetDecember 1, 2002
Tr. Severin
Calarasi
Bucuresti
Craiova
Constanta
Annual Cost 1 MEuro
Alexandria
Giurgiu
19Inhomogeneous Bandwidth Distributioin Latin
America. CAESAR Report (6/02)
Intl Links0.071 Gbps Used4,236 Gbps Capacity
Into Latin America
Need to Pay Attentionto End-point connections
20Recommendation Work to Close the Digital
Divide Bring the Needed Networks to All
Regions
- ICFA Members must work vigorously towards this
goal Locally, Nationally and
Internationally - Why ?
- Physicists from all world regions have the Right
to be full partners - It is the basis of our global community, and
our largest projects - Involvement of students, and outreach to the
community is vital to our field. In modern
times, this is founded on networks. - How ? We are the prototypical global
community - Developments by HENP of Grids, state-of-the-art
networks and systems for collaborative work on a
worldwide scale represent a unique opportunity,
for science and society - Work with SCIC other cognizant organizations
see following - And If We Dont ?
- We fail as a field, and as the first global
community in science
21Recommendation Work to Close the Digital Divide
- Identify Help Solve Technical or Coordination
Problems (1) Policy Problems - Lack of Competition Policies Restricting Market
Access, or
New Infrastructure Installation
High Market Prices 4 Vendor
Rule ? - Price Fixing Policies High BW subsidizes Low BW
- Lack of a Local or Regional Infrastructure
and/or No plan to develop one No Policy - (2) Coordination Problems
- Peering Transit Traffic Between Remote
Communities - (3) Knowledge Problems (Examples)
- What are the equipment and link technologies ?
what are the end-to-end prices vs. BW in major
markets ? - Dark Fiber Installation ? Operation ? Price ?
22Recommendation Work to Close theDigital Divide
(Several Perspectives)
- Work on Policies and Pricing Pk, In, Br, CN, SE
Europe, Africa - Find Ways to work with vendors, NRENs, and/or
Govts(e.g. Share pricing info. Encourage
Competition) - Help start and support Workshops on Networks,
Grids, and advanced applications - Help form regional support and training groups
for network and Grid developments, operations,
monitoring and troubleshooting - Propose Inter-Regional Programs Example (South
America) - NSF FIU Proposal (1/2003) EU _at_LIS CLARA
Proposal - Use Model Cases Convince Neighboring Countries
- Poland (to 5k km Fiber) Slovakia Czech Republic
- Exploit One-off Solutions E.g. extend the SILK
Project (DESY/FSU satellite links) to a SE
European site - Coordinate with Cognizant Organizations TERENA,
Internet2, AMPATH, APAN, IEEAF, WSIS, etc. to
help with solutions - Join Forces with Other Fields (e.g. Astrophysics)
23 Virtual Silk Highway Project Managed by
DESY and Partners
- Virtual SILK Highway Project (from 11/01) NATO
( 2.5 M) and Partners ( 1.1M) - Satellite Links to South Caucasus and
Central Asia (8 Countries) - In 2001-2 (pre-SILK) BW 64-512 kbps
- Proposed VSAT to get 10-50 X BW for same cost
- See www.silkproject.org
- Partners CISCO, DESY. GEANT, UNDP, US
State Dept., Worldbank, UC London, Univ.
Groenigen - SCIC Extend to a SE Europe Site ?
Note NATO Science for Peace Program
24Digital Divide Committee
25(No Transcript)
26CA-Tokyo by 6/03
NY-AMS Done 9/02
(Research)
27APAN Links in Asia January 2003
Japan-Korea Link 8 Mbps to 1 Gbps in January
2003IEEAF 10G 0.6G Links by June 2003
28 PingER (Also IEPM-BW)
Monitoring Sites
- Measurements from
- 38 monitors in 12 countries
- 790 remote hosts in 70 Countries 3500
monitor-remote site pairs - Measurements go back to Jan-95
- Reports on link reliability, quality
- Countries monitored
- Contain 78 of world population
- 99 of Internet users
Remote Sites
29 History Loss Quality
(Cottrell)
- Fewer sites have very poor to dreadful
performance - More have good performance(lt 1 Loss)
- BUT lt20 of the worldspopulation has Good
orAcceptable performance
30Recommendation Strongly Supportand Extend the
Monitoring Project
- This is Imperative, to
- Quantify and Bridge the Digital Divide
- To Do Continue and Extend the Monitoring
Coverageto All Countries - ICFA Assistance Find Sites and Contacts in
Albania, Armenia, Belarus, Bulgaria, Macedonia,
Yugoslavia, Costa Rica, Cuba, Pakistan 49 of 54
countries in Africa - Help identify Sources of Funding to Continue and
Extendthe Monitoring, and to Better Share
Information (Web Site) - For all countries with HENP programs
- Funding Agency Representatives are Asked to Help
- US DOE or NSF for IEPM at SLAC
- EU for ICTP that is working with IEPM for
Developing Nations
31Digital Divide Activities
- Questionnaire Distributed to Lab Directors
and Major Experiments - Plan on Project to Build HENP World Network
Map Updated and Maintained on a Web Site,
Backed by Database - Systematize and Track Needs and Status
- Share Information On
- Links Bandwidths Pricing Vendors
Technologies - Problems Overloading ( Where) Quality
Peering, etc. - Requirements Are They Being Met ?
- Identify Urgent Cases Focus on Opportunities to
Help - Propose 1st ICFA SCIC Workshop Focus on the
Digital Divide - Target Date February 2004 in Rio de Janeiro
(near LISHEP) - Organization Meeting July 2003
- Plan Statement at the WSIS, Geneva (December
2003) - Install and Leave Behind a Good Network
- Then 1 (to 2) Workshops Per Year, at Sites that
Need Help
32Digital Divide Sub-Committee Questionnaire
Response Extract
33World Summit on the Information Society(WSIS)
Geneva 12/2003 and Tunis in 2005
- The UN General Assembly adopted in 2001 a
resolution endorsing the organization of the
World Summit on the Information Society (WSIS),
to be convened under the patronage of the United
Nations Secretary-General, Kofi Annan, with the
International Telecommunication Union taking the
lead role in its preparation along with
interested UN organizations and the host
countries. - An Information Society A Common Definition,
Adoptedin the Tokyo Declaration of January
2003 One in which highly developed ICT
networks, equitable and ubiquitous access to
information, appropriate content in accessible
formats and effective communication can help
peopleachieve their potential - The Summit will offer a unique opportunity for
the world community to discuss and give shape to
the Information Society by bringing together
governments, international organizations, private
sector and civil society.
34World Summit on the Information Society(WSIS)
Geneva 12/2003 and Tunis in 2005
- In preparation for the Summit regional
conferences are organized to develop specific
proposals - Pan-European Regional Ministerial Conference,
November 7-9, 2002, Bucharest - Asia Regional Conference January 13-15, Tokyo
- Latin America and Caribbean Conference, January
29-31, Bavaro - Global Preparatory Meeting, February 17-28,
Geneva - CERN is preparing a Scientific Event in Geneva,
to highlightthe role of science in the formation
of an Information Society,shortly before the
Summit - Action Item for ICFA
- Prepare and Present a Statement to the World
Summit - For WSIS Schedules and Documents
http//www.state.gov/e/eb/cip/wsis
35WSIS-RO Role of New Technologies in Developing
the Information Society
- A Session at the Meeting in Bucharest, November
9, 2002 - Harvey Newman, Caltech
- Tony Hey, UK Core E-Science Program
- Peter Zangli, European Commission
- Hans Hoffmann, CERN Scientific and TT Director
- Fabrizio Gagliardi EDG Project Leader
- William Johnston, Lawrence Berkeley Lab
- HN for Dave Reese Tom West, CENIC, and Natl
Light Rail - Don Riley, CIO of Univ. of Maryland, CEO of
IEEAF - See http//cil.cern.ch8080/WSIS for documents
and presentations
36 Global Medical Research Exchange
Initiative Bio-Medicine and Health
Sciences
Global Quilt Initiative GMRE Initiative - 001
Propose Global Research and Education Exchange
for Physics
37ICFA and International Networking
- ICFA Statement on Communications in Intl
HEPCollaborations of October 17, 1996
See http//www.fnal.gov/directorate/icfa/icfa_comm
unicaes.html - ICFA urges that all countries and institutions
wishing to participate even more effectively and
fully in international HEP Collaborations should - Review their operating methods to ensure they
are fully adapted to remote participation - Strive to provide the necessary communications
facilities and adequate international bandwidth - ICFA Needs to Work with Experiments,
Governments, and Agencies, to Help Make this So
38SCIC Work in 2003
- Continue Digital Divide Focus
- Improve and Systematize Information in Europe
in Cooperation with TERENA and SERENATE - More in-depth information on Asia, with APAN
- More in-depth information on South America, with
AMPATH - Begin Work on Africa, with ICTP
- Set Up HENP Networks Web Site (Get Support, or
Funding) - Continue and if Possible Strengthen Monitoring
Work (IEPM) - Continue Work on Specific Improvements
- Brazil and South America, with RNP
- Russia
- Pakistan (?) India (?) China (?)
- Romania
- The World Summit on the Information Society
- Watch Requirements the Lambda Grid
Analysis revolutions - Discuss, Begin to Create a New Culture of
Collaboration
39Some Extra Slides Follow
40ICFA and Global Networks for HENP
- National and International Networks, with
sufficient (rapidly increasing) capacity and
capability, are essential for - The daily conduct of collaborative work in both
experiment and theory - Detector development construction on a global
scale Data analysis involving physicists from
all world regions - The formation of worldwide collaborations
- The conception, design and implementation of
next generation facilities as global networks - Collaborations on this scale would never have
been attempted, if they could not rely on
excellent networks
41ICFA Standing Committee on Interregional
Connectivity (SCIC)
- Created by ICFA in July 1998 in Vancouver
Following ICFA-NTF - CHARGE
- Make recommendations to ICFA concerning the
connectivity between the Americas, Asia and
Europe (and network requirements of HENP) - As part of the process of developing
theserecommendations, the committee should - Monitor traffic
- Keep track of technology developments
- Periodically review forecasts of future
bandwidth needs, and - Provide early warning of potential problems
- Create subcommittees when necessary to meet the
charge - The chair of the committee should report to ICFA
once peryear, at its joint meeting with
laboratory directors (Today) - Representatives Major labs, ECFA, ACFA, NA
Users, S. America
42ICFA SCIC RE Backbone and International Link
Progress
- GEANT Pan-European Backbone (http//www.dante.net/
geant) - Now interconnects gt31 countries many trunks 2.5
and 10 Gbps - UK SuperJANET Core at 10 Gbps
- 2.5 Gbps NY-London, with 622 Mbps to ESnet and
Abilene - France (IN2P3) 2.5 Gbps RENATER3 backbone from
October 2002 - Lyon-CERN Link Upgraded to 1 Gbps Ethernet
- Proposal for dark fiber to CERN by end 2003
- SuperSINET (Japan) 10 Gbps IP and 10 Gbps
Wavelength Core - Tokyo to NY Links 2 X 2.5 Gbps started Peer
with ESNet by Feb. - CAnet4 (Canada) Interconnect customer-owned
dark fiber nets across Canada at 10 Gbps,
started July 2002 - Lambda-Grids by 2004-5
- GWIN (Germany) 2.5-10 Gbps Core Connect to US
at 2 X 2.5 GbpsSupport for SILK Project
Satellite links to FSU Republics - Russia 155 Mbps Links to Moscow (Typ. 30-45 Mbps
for Science) - Moscow-Starlight Link to 155 Mbps (US NSF
Russia Support) - Moscow-GEANT -Stockholm Links 155 Mbps Plan for
622 Mbps
43RE Backbone and Intl Link Progress
- Abilene (Internet2) Upgrade from 2.5 to 10 Gbps
started in 2002 - Encourage high throughput use for targeted
applications FAST - ESNET Upgrade to 10 Gbps As Soon as Possible
- US-CERN
- to 622 Mbps in August Move to STARLIGHT
- 2.5G Research Triangle from 8/02
STARLIGHT-CERN-NL to 10G in 2003. 10Gbps
SNV-Starlight Link Loan from Level(3) - SLAC IN2P3 (BaBar)
- Typically 400 Mbps throughput on US-CERN,
Renater links - 600 Mbps Throughput is BaBar Target for Early
2003 (with ESnet and Upgrade) - FNAL ESnet Link Upgraded to 622 Mbps
- Plans for dark fiber to STARLIGHT, proceeding
- IEEAF Donations NY-AMS Done Sept. 2002
US-JP-KR-CN-SG By June 2003 Gbps. 622 Mps 10G
Research Wavelength - US National Light Rail Proceeding Startup
Expected this Year
44APAN Links in January 2003
45RNP Brazil
FIU Miami from So. America
Note Auger (AG), ALMA (Chile), CMS-Tier1 (Brazil)
46Infrastructure to Latin America
- Miami is the hub to 9 Major submarine fiber cable
routes - Americas 12
- Global Crossing
- Columbus II III
- Telefonica Emergia/SAM 1
- Arcos 1
- Maya 1
- 360 Americas
- Miami is ranked 8th in the top 50 Inter-Regional
Internet hub cities
Source Packet Geography, 2002
Julio Ibarra
47Undersea Optical Infrastructure
The total aggregate bandwidth capacity Latin
America and Caribbean region is estimated at
4,236 GB
48NRNs Bandwidth in Latin America
Source CAESAR - Review of Develop-ments in Latin
America
49It is generally accepted that once a technology
is perceived as having broad utilitarian value,
price as a of per capita income, is the main
driver of penetration
C. Casasus, CUDI (Mexico) CANARIE
50Penetration of telecommunications in low income
countries is further inhibited by at least 3
factors
- Low income per capita
- Less competition. Higher prices from monopolies
- Fewer applications. No broad utilitarian value
C. Casasus, CUDI (Mexico) CANARIE
51Income vs. penetration given the price of a
technology
Per capita Income
High
Low income countries
Penetration
Low
Low
High
C. Casasus, CUDI (Mexico) CANARIE
52Telecom monopolies have even higher prices in low
income countries
- Fewer entrants. Less competition
- No unbundling
- Price cap regulation creates cross subsidies
between costumer groups. - Large customers (inelastic) subsidize small
costumers (elastic). High bandwidth services are
very expensive - Inefficient ROW regulation
- Inefficient spectrum policies
C. Casasus, CUDI (Mexico) CANARIE
53Dai Davies SERENATE Workshop Feb. 5, 2003
54Relative Cost of Connectivity Compared with
Number of Suppliers
55Multipliers for Differing Circuit Speeds
56Jensen, ICTP
57Limited by many externalsystemic
factorsElectricityImport DutiesEducation
Trade restrictions
Jensen, ICTP
58FAST (Caltech) A Scalable, Fair Protocol for
Next-Generation Networks from 0.1 To 100 Gbps
SC2002 11/02
Highlights of FAST TCP
- Standard Packet Size
- 940 Mbps single flow/GE card
- 9.4 petabit-m/sec
- 1.9 times LSR
- 9.4 Gbps with 10 flows
- 37.0 petabit-m/sec
- 6.9 times LSR
- 22 TB in 6 hours in 10 flows
- Implementation
- Sender-side (only) mods
- Delay (RTT) based
- Stabilized Vegas
Sunnyvale-Geneva
Baltimore-Geneva
Baltimore-Sunnyvale
SC2002 10 flows
SC2002 2 flows
I2 LSR
29.3.00 multiple
SC2002 1 flow
9.4.02 1 flow
22.8.02 IPv6
URL netlab.caltech.edu/FAST
Next 10GbE 1 GB/sec disk to disk
C. Jin, D. Wei, S. Low FAST Team Partners
59Aggregate throughput
88
90
- FAST
- Standard MTU
- Utilization averaged over gt 1hr
90
Average utilization
92
95
1 GbE flow 2 flows 7 flows
9 flows 10 flows (8.8Gbps)