ICFA SCIC Meeting May 27 2003

1

• ICFA SCIC Meeting May 27 2003

Harvey B. Newman California Institute
of Technology
2
ICFA SCICMay 27, 2003 Agenda

• 730 Startup Test VRVS Connections
• 800 Introduction
  H. Newman
• 820 Brief Status Reports from Sites and
  Working Groups ALL
• 1000 BREAK
• 1015 Schedule for Digital Divide Conference in
  Rio A. Santoro Schedule
  for Organization Meeting at CERN in July
• 1100 Plans for Advanced Technologies WG Report
  by Summer 2003
• 1115 Discussion of Work for the Rest of 2003
• Next Report to ICFA Directors Meeting in
  Paris (February 14-15 2004)
• 1145 AOB Next Meeting
• 1200 ADJOURN

3
SCIC in 2002-3A Period of Intensive Activity

• Web Page http//cern.ch/ICFA-SCIC/
• Monitoring Les Cottrell (SLAC)
  (http//www.slac.stanford.edu/xorg/icfa/scic-netm
  on) With Richard Hughes-Jones (Manchester),
  Sergio Novaes (Sao Paolo) Sergei Berezhnev
  (RUHEP), Fukuko Yuasa (KEK), Daniel Davids
  (CERN), Sylvain Ravot (Caltech), Shawn McKee
  (Michigan) Report Today New Data With ICTP
• Advanced Technologies R. Hughes-Jones, Olivier
  Martin (CERN) With Vladimir Korenkov (JINR,
  Dubna), H. Newman Report this Summer
• The Digital Divide Alberto Santoro (Rio, Brazil)
  
• With V. Ilyin (MSU), Y. Karita(KEK), D.O.
  Williams (CERN)
• Also V. White (FNAL), J. Ibarra and H. Alvarez
  (AMPATH),D. Son (Korea), H. Hoorani, S. Zaidi
  (Pakistan), S. Banerjee (India)
• Digital Divide Workshop in February 2004 Focus
  on Actions
• Key Requirements Harvey Newman and Charlie Young
  (SLAC)
• New Roadmap

4
ICFA and International Networking

• ICFA Statement on Communications in Intl
  HEPCollaborations of October 17, 1996
  See http//www.fnal.gov/directorate/icfa/icfa_comm
  unicaes.html
• ICFA urges that all countries and institutions
  wishing to participate even more effectively and
  fully in international HEP Collaborations should
• Review their operating methods to ensure they
  are fully adapted to remote participation
• Strive to provide the necessary communications
  facilities and adequate international bandwidth
  

Action Item Create a Culture of
Collaboration Starting with Collaboration and
Lab Management
5
ICFA Standing Committee on Interregional
Connectivity (SCIC)

• Created by ICFA in July 1998 in Vancouver
  Following ICFA-NTF
• CHARGE
• Make recommendations to ICFA concerning the
  connectivity between the Americas, Asia and
  Europe (and network requirements of HENP)
• As part of the process of developing
  theserecommendations, the committee should
• Monitor traffic
• Keep track of technology developments
• Periodically review forecasts of future
  bandwidth needs, and
• Provide early warning of potential problems
• Create subcommittees when necessary to meet the
  charge
• Representatives Major labs, ECFA, ACFA, NA
  Users, S. America
• The chair of the committee should report to ICFA
  once peryear, at its joint meeting with
  laboratory directors (Feb. 2004)

6
Rapid Network Advances and the Digital Divide

• The current generation of 2.5-10 Gbps network
  backbones arrived in the last 15 Months in the
  US, Europe and Japan
• Major transoceanic links also are reaching 2.5 -
  10 Gbps
• Capability Increased 4 Times, i.e. 2-3 Times
  Moores Law
• This is a direct result of the continued
  precipitous fall of network prices for 2.5 or 10
  Gbps in these regions
• Higher prices remain in the poorer regions
• There are strong prospects for further advances
  that will cause the Divide to become a Chasm,
  Unless We Act For the Rich Regions
• 10GigE in campusmetro backbones GigE/10GigE to
  desktops ? 40 GigE within 2-3 Years ?
• Advances in protocols (TCP) to use networks at
  1-10 Gbps
• DWDM More 10G wavelengths and/or 40G speeds on
  a fiber
• Owned or leased wavelengths in the last mile,
  the region, and/or across the country

7
2003 NSF ITRs Globally EnabledAnalysis
Communities Collaboratories

• Develop and build Dynamic Workspaces
• Construct Autonomous Communities Operating
  Within Global Collaborations
• Build Private Grids to support scientific
  analysis communities
• e.g. Using Agent Based Peer-to-peer Web
  Services
• Drive the democratization of science via the
  deployment of new technologies
• Empower small groups of scientists (Teachers
  and Students) to profit from and contribute
  to intl big science

8
HENP Lambda GridsFibers for Physics

• Problem Extract Small Data Subsets of 1 to 100
  Terabytes from 1 to 1000 Petabyte Data Stores
• Survivability of the HENP Global Grid System,
  with hundreds of such transactions per day
  (circa 2007)requires that each transaction be
  completed in a relatively short time.
• Example Take 800 secs to complete the
  transaction. Then
• Transaction Size (TB) Net
  Throughput (Gbps)
• 1
  10
• 10
  100
• 100
  1000 (Capacity of
  Fiber
  Today)
• Summary Providing Switching of 10 Gbps
  wavelengthswithin 3-5 years and Terabit
  Switching within 5-8 yearswould enable
  Petascale Grids with Terabyte transactions,to
  fully realize the discovery potential of major
  HENP programs, as well as other data-intensive
  fields.

9
FAST TCP Baltimore/Sunnyvale
88

• RTT estimation fine-grain timer
• Fast convergence to equilibrium
• Delay monitoring in equilibrium
• Pacing reducing burstiness

10G
90
9G
90

• Measurements
• Std Packet Size
• Utilization averaged over gt 1hr
• 3000 km Path

Average utilization
92
8.6 Gbps 21.6 TB in 6 Hours
95
Fair SharingFast Recovery
1 flow 2 flows 7 flows
9 flows 10 flows
10
On Feb. 27-28, a Terabyte of data was transferred
in 3700 seconds by S. Ravot of Caltech between
the Level3 PoP in Sunnyvale near SLAC and CERN
through the TeraGrid router at StarLight from
memory to memory As a single TCP/IP stream at
average rate of 2.38 Gbps. (Using large
windows and 9kB Jumbo frames)This beat the
former record by a factor of 2.5, and used
the US-CERN link at 99 efficiency.
I2 LSR 10GigE Transfers with GridDT
European Commission
10GigE NIC
11
National Light Rail Footprint

• NLR
• Buildout Started November 2002
• Initially 4 10 Gb Wavelengths
• To 40 10Gb Waves in Future

• Transition beginning now to optical,
  multi-wavelength RE networks.
• Also Note IEEAF/GEO plan for dark fiber in
  Europe

12
Optical Packet Routing Using ? ConversionD.
Blumenthal, UC Santa Barbara

• Optical gt Electronic Switching
• Microprocessor Power
• Per Fiber Capacity Increases

Wavelength Router
Fast Wavelength Converter
Packet switched to wavelength ?5
Packet switched to wavelength ?2
Packets at Wavelength ?1 and ?7
Control Signals
Fast Tunable Laser

• Circuit Switched Mode
• Burst Mode
• Packet Mode

13
80 Gbps Optical Packet Routing with Label
Swapping Results (UCSB)
14
HENP Major Links Bandwidth Roadmap (Scenario)
in Gbps
Continuing the Trend 1000 Times Bandwidth
Growth Per DecadeWe are Rapidly Learning to Use
Multi-Gbps Networks Dynamically
15
ICFA Network Task Force 1998 Bandwidth Reqments
Projection (Mbps)
NTF
1001000 X Bandwidth Increase Foreseen for
1998-2005 See the ICFA-NTF Requirements
Report http//l3www.cern.ch/newman/icfareq98.htm
l
16
Bandwidth Growth of Global HENP Networks

• Rate of Progress gtgt Moores Law. (US-CERN
  Example)
• 9.6 kbps Analog (1985)
• 64-256 kbps Digital (1989 - 1994)
  X 7 27
• 1.5 Mbps Shared (1990-3 IBM)
  X 160
• 2 -4 Mbps (1996-1998) X
  200-400
• 12-20 Mbps (1999-2000)
  X 1.2k-2k
• 155-310 Mbps (2001-2)
  X 16k 32k
• 622 Mbps (2002-3)
  X 65k
• 2.5 Gbps ? (2003-4)
  X 250k
• 10 Gbps ? (2005)
  X 1M
• A factor of 1M over a period of 1985-2005 (a
  factor of 5k during 1995-2005)
• HENP has become a leading applications driver,
  and also a co-developer of global networks

17
UltraLight An Ultra-scale Optical Network
Laboratory for Next Generation Science

• Caltech, UF, FIU, UMich, SLAC,FNAL,MIT/Haysta
  ck,CERN, UERJ(Rio), NLR, CENIC,
  UCAID,Translight, UKLight, Netherlight, UvA,
  UCLondon, KEK, Taiwan
• Cisco, Level(3)

18
IEPMHistory Loss Quality (Cottrell)

• Fewer sites have very poor to dreadful
  performance
• More have good performance(lt 1 Loss)

• BUT lt20 of the worldspopulation has Good
  orAcceptable performance

19
Work on the Digital DivideSeveral Perspectives

• Identify Help Solve Technical Problems
  Natl, Regional, Last 10/1/0.1 km Peering.
• SCIC Questionnaire to Experiment Managements Lab
  Directors
• Strong Support for Monitoring Projects, such as
  IEPM
• Inter-Regional Proposals (Example Brazil)
• US NSF CHEPREO (PEP 5/2003) EU _at_LIS Proposal
• Work on Policies and/or Pricing pk, in, br, cn,
  SE Europe,
• Find Ways to work with vendors, NRENs, and/or
  Govts
• Use Model Cases Installation of new advanced
  fiber infrastructures Convince Neighboring
  Countries
• Slovakia Poland (to 5k km fiber)
• Exploit One-off Solutions E.g. the Virtual SILK
  Highway Project (DESY/FSU satellite links)
  Extend to a SE European site
• Work with Other Cognizant Organizations Terena,
  Internet2, AMPATH, IEEAF, UN, GGF, etc. help
  with technical and/or political solns

20
Telecom monopolies have even higher prices in low
income countries

• Fewer Market Entrants. Less Competition
• Lower Income ? Less Penetration of New
  Technologies
• Price cap regulation creates cross subsidies
  between costumer groups.
• Large customers (inelastic demand) subsidize
  small costumers (elastic) High bandwidth
  services are very expensive
• Inefficient Rights of Way (ROW) regulation
• Inefficient spectrum allocation policies

C. Casasus, CUDI (Mexico) W. St. Arnaud, CANARIE
(Canada)
21
D. Davies SERENATE Workshop Feb. 5, 2003 Next
June 15-17
22
APAN Links in Asia January 2003


Typical Intra-Asia Intl Links 0.5 45 Mbps
Progress Japan-Korea Link 8 Mbps to 1 Gbps in
Jan. 2003IEEAF 10G 0.6G Links by June 2003
23
Inhomogeneous Bandwidth Distributionin Latin
America. CAESAR Report (6/02)



J. Ibarra, AMPATH Wkshp
Need to Pay Attentionto End-point
connections (e.g. UERJ Rio)
In Progress622 Mbps MiamiRio CLARA Project
Brazil, Mexico, Chile, Agentina
Intl Links4,236 Gbps Fiber Capacity Into
Latin America Only 0.071 Gbps Used
24
Cultivate and promote practical solutions to
delivering scalable, universally available and
equitable access to suitable bandwidth and
necessary network resources in support of
research and education collaborations.
http//www.ieeaf.org
Groningen Carrier Hotel
TransAtlantic, Transpacific, Intra-US and
European Initiatives
25
US-JP-KR-CN-SG Tokyo by 6/03 ?
NY-AMS Done 9/02
(Research)
26
Jensen, ICTP
Typ. 0-7 bpsPer Person
27
SCIC Work in 2003

• Continue Digital Divide Focus
• Improve and Systematize Information in Europe
  in Cooperation with TERENA and SERENATE
• More in-depth information on Asia, with APAN
• More in-depth information on South America, with
  AMPATH
• Begin Work on Africa, with ICTP
• Set Up HENP Networks Web Site and Database
• Share Information on Problems, Pricing Example
  Solutions
• Continue and if Possible Strengthen Monitoring
  Work (IEPM)
• Continue Work on Specific Improvements
• Brazil and So. America Romania Russia India
  Pakistan, China
• An ICFA-Sponsored Statement at the World Summit
  on the Information Society (12/03 in Geneva),
  prepared by SCIC CERN
• Watch Requirements the Lambda Grid
  Analysis revolutions
• Discuss, Begin to Create a New Culture of
  Collaboration

28
Some Extra Slides Follow
29
Global Networks for HENPCirca 2003

• National and International Networks, with
  sufficient (and rapidly increasing) capacity and
  capability, are essential for
• Data analysis, and the daily conduct of
  collaborative work in both experiment and
  theory, Involving physicists from all world
  regions
• Detector development construction on a global
  scale
• The formation of worldwide collaborations
• The conception, design and implementation of
  next generation facilities as global networks
• Collaborations on this scale would never have
  been attempted, if they could not rely on
  excellent networks (L. Price)

30
SCIC in 2002-3A Period of Intense Activity

• Formed WGs in March 2002 9 Meetings in 12 Months
• Strong Focus on the Digital Divide
• Presentations at Meetings and Workshops(e.g.
  LISHEP, APAN, AMPATH, ICTP and ICFA Seminars)
• HENP more visible to governments in the WSIS
  Process
• Five Reports Presented to ICFA Feb. 13,2003See
  http//cern.ch/icfa-scic
• Main Report Networking for HENP H. Newman et
  al.
• Monitoring WG Report L. Cottrell
  
• Advanced Technologies WG Report R. Hughes-Jones,
  
  O. Martin et al.
• Digital Divide Report A. Santoro et al.
• Digital Divide in Russia Report V. Ilyin

31
HENP Networks Status and Outlook SCIC General
Conclusions

• The scale and capability of networks, their
  pervasiveness and range of applications in
  everyday life, and HENPs dependence on networks
  for its research, are all increasing rapidly.
• However, as the pace of network advances
  continues to accelerate, the gap between the
  economically favored regions and the rest of
  the world is in danger of widening.
• We must therefore work to Close the Digital
  Divide
• To make Physicists from All World Regions Full
  Partners in Their Experiments and in the Process
  of Discovery
• This is essential for the health of our global
  experimental collaborations, our plans for
  future projects, and our field.

32
Global Medical Research Exchange
Initiative Bio-Medicine and Health
Sciences
2002-3 Beginning a Plan for a Global Research
and Education Exchange for High Energy Physics
Global Quilt Initiative GMRE Initiative - 001
33
Limited by many external systemic
factors Electricity Import
Duties Education Trade restrictions
Progress in Africa ?
Jensen, ICTP
34
Networks, Grids and HENP

• Current generation of 2.5-10 Gbps backbones and
  intl links arrived in the last 15 Months in
  the US, Europe and Japan
• Capability Increased 4 Times, i.e. 2-3 Times
  Moores
• Reliable high End-to-end Performance of network
  applications is required (large transfers
  Grids), and is achievable
• Achieving this more broadly for HENP requires
• End-to-end monitoring a coherent approach (IEPM
  Project)
• Getting high performance (TCP) toolkits in
  users hands
• Isolating and addressing specific problems
• Removing Regional, Last Mile Bottlenecks and
  Compromises in Network Quality are now On the
  critical path, in all world regions
• Digital Divide Network improvements are
  especially needed in SE Europe, So. America SE
  Asia, and Africa
• Work in Concert with Internet2, Terena, APAN,
  AMPATH DataTAG, the Grid projects and the
  Global Grid Forum