Reconstruction production

1
Reconstruction production

• Y. Fisyak, BNL

2
What is needed to run athena ?

• To define script to install a working directory
  and JobOptionFile.txt
• To define a bunch of environment variables
  depending on ATLAS release
• PATH and LD_LIBRARY_PATH to find executable(s)
  and shared libraries
• 650 packageROOT variables to handle
  constructions
• include "ATHENACOMMONROOT/share/Atlas_ZebraTDR.U
  nixStandardJob.txt
• in JobOptionFile.txt and
• constructions like
• cp -f ATHENACOMMONROOT/share/slug.car .
• in shell script during installation of working
  directory
• To define input and output files

3
How is it defined in production?

• In current production (HLT with 6.0.4 release)
  this is done via athena_job perl script (see
  http//www.usatlas.bnl.gov/fisyak/atlas/Productio
  n/)
• A typical job looks like
• perl athena_job
• i dc1.002001.lumi10.05259.hlt.pythia_jet_25.zeb
  ra
• -o ../dc1.002001.lumi10.recon.010._05259.hlt.pyt
  hia_jet_25.hbook
• -w /usatlas/data04/dc1/work/6.0.4
• -s recon.gen.v7.bnl
• -j trigger.lumi10.noise.604.job
• It defines input (-i) and output (-o) files,
  working directory (-w), script to install the
  working directory (-s), and jobOptionFile (-j)

4
How is it defined (cont.)

• The perl script uses a set of default environment
  variables
• setenv ATLASDATA /afs/usatlas.bnl.gov/offline/data
  
• setenv ATLAS_BASE_RELEASE 6.0.4
• setenv ATLAS_ROOT /afs/usatlas.bnl.gov
• setenv CERNSW cernsw
• setenv CMTCONFIG i686-rh73-gcc295
• setenv CMTSITE BNL
• setenv CMTVERS v1r12p20020606
• setenv DIST /afs/usatlas.bnl.gov/software/dist
• setenv TEST_RELEASE_DIR /afs/usatlas.bnl.gov/users
  /fisyak/public/prod/6.0.4
• setenv CMTROOT /afs/usatlas.bnl.gov/cernsw/contrib
  /CMT/v1r12p20020606
• setenv CMTPATH /afs/usatlas.bnl.gov/users/fisyak/p
  ublic/prod/6.0.4
• setenv DISTREL /afs/usatlas.bnl.gov/software/dist/
  6.0.4
• setenv ATLASBIN /afs/usatlas.bnl.gov/software/dist
  /6.0.4/InstallArea/i686-rh73-gcc295/bin
• Any of these environment variables can be
  redefined
• athena_job ATLAS_BASE_RELEASE6.3.0

5
What athena_job does ?

• Create working directory
• Create in this directory requirements file and
  run a particular version of cmt with particular
  version of AtlasSettings (the versions of cmt and
  AtlasSettings are depended on ATLAS_BASE_RELEASE)
• Create soft links and copy data files in the
  working directory
• Find input file using magda (magda_getfile) on
  disks or HPSS
• Run athena with the JobOptionFile.txt
• Save output file (ntuple.hbook)
• Remove the working directory
• Release input file (magda_releasefile)

6
Bookkeeping

• Bookkeeping is done with one MySQL table
  (Production.jobdata) in which it is tracked
  transformation through the steps
• DEFINED (registration of the output file),
• lsf submit status (PEND, RUN, SSUSP),
• ABORTED (job fails in lsf),
• UNDECIDED (job finished successfully, output file
  has to be validated),
• FAILED (job finished successfully but output
  file is corrupted),
• VALIDATED .

7
Mini productions (PWG/user oriented).

• Two types of production outside of DC chain
• Physics Working Group (PWG) production with
  significant consumption of resources
• It should be managed via USATLAS computing,
• It can be done using the above scheme with
  tracking all modification of environment with
  respect to official DC production in Virtual
  Data Catalog
• Personal (user) production with resources which
  are not collided with DC and PWG productions.
• This production requires adjustments of above
  scheme to user environment
• It is not clear what level of bookkeeping this
  production is necessary (MySQL versus flat files)