TRIUMF a TIER 1 Center for ATLAS Canada

1
TRIUMF a TIER 1 Center for ATLAS Canada

• Steven McDonald
• TRIUMF Network Computing Services
• steven.mcdonald_at_triumf.ca

iGrid 2005 San Diego Sept 26th 29th 2005
2












3
The TRIUMF Facility
Canadas National Laboratory for sub-atomic
physics

• Located in Vancouver, Canada
• Research is performed locally using the
• worlds largest 500MeV Cyclotron

• As well as internationally at
• CERN-ATLAS
• BNL-KOPIO
• KEK-Neutrino Experiment
• 400 full-time employees

• Nuclear Physics
• Particle Physics
• Astrophysics
• Material Science

• Proton PET Therapy
• Radiopharmaceuticals
• Radioactive beams
• Microchip design

4
Politics Funding

• TRIUMF 5 yr budget cycle runs 2005-2009, TRIUMF
  ATLAS Tier1 centre is one of the main TRIUMF
  programs to be undertaken in that 5 year plan
  
• However, the Canadian political climate has been
  somewhat turbulent for the last year with an
  election expected to be called at the end of the
  year, consequently funding for ATLAS at TRIUMF
  has not been fully secured?
• TRIUMFs 5 yr budget request for an ATLAS Tier1
  centre was 8M Capital, with 9 more additional
  FTEs
  
• Two FTEs were hired last year for ATLAS at
  TRIUMF
  
• Until Funding is fully secured TRIUMFs Core
  Computing Networking Group is providing the
  initial 10G infrastructure, along with CANARIE
  and HEPnet Canada.

5
The ATLAS Experiment

• 2000 physicists in 34 countries
• ATLAS Canada 0.6 TIER1
• 80 Physicists
• 4.5 of ATLAS Colab.
• Size of 5 story Building
• millions of channels of data
• Discover the Higgs particle
• 30 million events per second

Tier0 Tier1 network meeting in Amsterdam Jan
05 made it clear all Tier1 sites would require a
10 Gbps network connection to CERN
This would accommodate the expected sustained da
ta rates with head room for efficiency, burst and
recovery traffic.
Totals for all 10 TIER1s integrated over the
years
1 kSI2K 3GHz CPU
6
3rd week Sept 05
Amsterdam
Sept 1 05
End Oct 05
7
The TRIUMF-CERN 1GbE Lightpath(s)

• 1 GbE circuit
• established April 18th 2005
• 2nd GbE circuit
• established July 19th 2005

• TRIUMF
• BCNET
• CANARIE
• SURFnet
• CERN

8
ATLAS Tier2 in Canada
There will be two Tier 2s Centres in Canada,
one in the East and one in the West. Each centre
will be a distributed across several sites, with
one contact point
Western Tier2
Eastern Tier2
1G
1G
10G
CERN TIER 0
TRIUMF Tier 1
Tier2s
Dedicated e2e Lightpaths
Can be lightpaths or on the routed 1Gbps CAnet
4
research network
5G ?
? Tier 1
9
CWDM Lightpaths at TRIUMF
Aug 2002
/CAnet4



TRIUMF Local loop to Gigapop
CAnet4 PoP 22km
10 GbE _at_1550nm
2x 1 GbE _at_1530nm
2x 1 GbE _at_1510nm
2x 1 GbE _at_1690nm
10
TRIUMF WAN CWDM Equipment
Single Pair Fiber
MRV CWDM

• 4 1GbE channels
• Passport 8600
• ORAN
• WESTGRID
• 2x CERN

BCNET 22km
Potential to Add 2 more 1GbE channels
10GbE Foundry Switch - CERN
2x GbE TDM
4 Port Optical Mux
SFP
11
Transfer results over 10GbE circuit
12
TRIUMF Tier1 ? Canada Tier2s
Participating Canadian Tier2 sites in SC3
Simon Fraser University University of Victoria U
niversity of Alberta
University of Toronto
T1-T2 network activity Aug 17 - Aug 24 All four
Tier 2 sites participating
13
A Daily snap-shot of SC3
14
10 GbE Lightpaths to CERN 2004-05
CERN
TRIUMF
v
v
v
X
Atlantic Crossing
v
v
v
15
Summary

• Currently,
• 2 x 1 GbE circuits TRIUMF to CERN (since April
  18th / July 19th 2005)
  
• 10 GbE Foundry NI40Gs NI1500 on loan

• In Progress To be completed by end of year
• Circuits
• Sept 1st 2005 CANARIE/CAnet4 was delivered a
  10GbE circuit between Amsterdam and New York
  
• 3rd week Sept 2005 CAnet4 add 4th OC192 circuit
  across Canada
  
• End of October add a 5th OC192 circuit across
  Canada
  
• Still waiting on final Lambda between
  Amsterdam-CERN (GEANT2 ?)
  
• 10 GbE Equipment
• Last mile DWDM optics for TRIUMF/BCNET expected
  by Oct/Nov 05
  
• 10G Foundry Rx-4 switch ordered, expect delivery
  Oct/Nov 05