Exporting Raw/ESD data from Tier-0 ? Tier-1s

1
Exporting Raw/ESD data from Tier-0 ? Tier-1s

• Wrap-up

2
The problem (Bernd)

• One copy of the LHC raw data for each of the LHC
  experiments is shared among the Tier-1s
• Full copies of the ESD data (1/2 of raw data
  size)
• Total 10PB/year exported from CERN
• The full machinery for doing this automatically
  should be in place for full-scale tests in 2006

3
Tier-1 resources
TRIUMF 2 machines purchased, 1Gbit(?)
RAL Gbit link at present Parallel activities from ATLAS and CMS Not enough effort to dedicate for the moment More hardware in September
FNAL Just finished CMS DC very labor intensive Enough resources to sustain 2TB/day
GridKA 1Gbit at present, expanding to 10Gbit in October/November Storage system is ready (dCache TSM)
BNL SRM service almost ready (in a month) One gridftp node OC12 connection, not much used
NIKHEF/SARA 10Gbit since more than a year Running data challenges for experiments but mainly CPU intensive
IN2P3/Lyon Not yet ready with interface to MSS 1Gbit
4
Agreed tests

• Simple disk-to-disk, peer-to-peer
• Simple disk-to-disk, one-to-many
• MSS-to-MSS
• In parallel?
• Transfer scheduling
• Replica catalogue management

5
Test 1 Simple test

• disk to disk gridftp or bbftp (loadbalanced)
• Aggregated peer-to-peer transfers
• Files written, deleted and rewritten (process
  driven from CERN)
• 3 weeks preparation
• Goal 1 week running without intervention
• Tier-1 responsibility tune gridftp headnodes for
  best transfer performance

6
Test 2 simple but one-to-many

• Additional Tier-0 complexity export data to more
  than one Tier-1 in parallel

7
Test 3 slide in MSS

• Issue with SRM interoperability
• GGF/GSM WG initiative to define a SRMv1.1
  evolution SRM-basic
• Standardized error reporting

8
Test 4/5 (parallel)

• Test 4 slot in transfer scheduling a scheduling
  service controls the replication
• Test 5 add Replica catalogue and management
  services

9
Timescales
TRIUMF RAL
Transfer scheduling
one-to-many?
BNL
FNAL
SRM-Basic ready(?)
10
Next steps

• Statements from the other Tier-1s
• NIKHEF, GridKA, Lyon
• PIC, CNAF, others?
• Who is driving/coordinating? site contacts?
• Meetings ?
• Speed up SRM-basic specification process
• ...