BaBar Experience

1
BaBar Experience

• Tim Adye
• Rutherford Appleton Laboratory
• PPNCG Meeting
• Brighton
• 11th September 2002

2
Talk Outline

• New (more distributed) BaBar computing model
• and its impact, especially on RAL
• User experience
• Transfers SLAC-gtRAL
• Transfers UK-gtSLAC

3
New Computing Model

• Goal is to spread computing load much more around
  the collaboration
• Simulation production is already highly
  distributed
• Small-scale analysis already performed
  atUniversities (9 in UK) and Regional Centres
  (eg. RAL)
• Now have three new Tier A centres
• Lyon Objectivity (database) analysis (since
  last year)
• RAL Kanga (ROOT MicroDST) analysis (from May
  02)
• Padova Reprocessing (commissioning)
• RAL has relieved SLAC of all Kanga analysis
• Each site requires large data transfers from and
  to SLAC

4
BaBar CPU Usage at RAL Tier A
5
User Experience

• Now have users throughout US and Europe
• Interactive experience is generally excellent
• Connecting to RAL and working at RAL was very
  fast, as fast as at SLAC.
• Uriel Nauenberg, University of Colorado at
  Boulder.
• AFS access between UK and SLAC is still slow
• SLAC -gt RAL AFS, RALUK -gt SLAC AFS
• Maybe an intrinsic property of AFS with slower RTT

6
Bulk transfers SLAC-gtRAL

• Production at SLAC has come in bursts
• Reprocessing old data
• Kanga production is the final (and relatively
  simple) step in a long processing chain
• We have had no problem keeping up with steady
  state
• Sometimes delays of 1-2 weeks to catch up with a
  burst in production
• This is quite acceptable
• So far, gt19 TB copied from SLAC and on disk at
  RAL
• 15 TB since January
• RAL is now the primary Kanga repository, so
  others in UK/Europe/US will copy from us
• So far modest most ltlt 1TB per site (10-20 sites)

7
Kanga Data transfersbbftp SLAC-gtRAL
8
Transfer Rateper bbftp session (2-20 streams
each)
9
Bandwidth

• Transfer rate of 5-25 Mbit/s is obviously much
  less than SLAClt-gtRAL 622 Mbit/s link
• Actually get up to 50 Mbit/s by using multiple
  sessions (on different servers)
• Probably several effects limit us
• Only run a few (1-5) simultaneous sessions
• More is cumbersome to manage
• Currently use only a couple of 100 Mbit/s servers
  at RAL
• Firewall problems with bbftp limit servers we can
  use
• Will soon have dedicated import/export Gbit
  servers
• bbftp doesnt handle small files very efficiently
• Typical file sizes 10-500 MB
• Will use bbcp, but requires a bug fix (on Andys
  list)
• Not a problem at the moment

10
Transfers UK-gtSLAC

• UK now performs 75 of total BaBar Simulation
  Production
• Mostly at University sites, though RAL is
  starting up
• Objectivity output files sent back to SLAC
• Size reduced by a factor of 8 after dropping
  intermediate files
• no longer required for most analysis
• Output then skimmed, converted, and re-exported
  to Tier A and C sites

11
Simulation Transfers to SLAC (all sites)
12
UK Simulation ProductionA typical UK site

• Transfer rate limited by localSLAC
  infrastructure and number of bbftp streams
  (currently 3)
• Gaps in transfers show we are keeping up CPU is
  main limit
• Full plots at http//www.ep.ph.bham.ac.uk/user/smi
  th/

Transfer rate
13
Summary

• New computing model is heavily reliant on network
• Especially UK to/from SLAC
• User experience is good
• but is there anything we can do about AFS?
• Bulk transfer from and to SLAC currently limited
  by local infrastructure
• Nevertheless, we are easily keeping up with SLAC
  and UK farm production