Geant4 Physics Validation: Use of the GRID Resources

1
Geant4 Physics Validation Use of the GRID
Resources

• Patricia Mendez Lorenzo
• CERN (IT-GD) / CNAF
• Geant4 Bio-Medical Developments
• Geant4 Physics Validation
• INFN Genova, 13-20 July 2005

2
Outlook

• Introduction to the LCG
• Geant4 in LCG
• First Geant4 Productions
• Results and Summary
• Future Plans

3
What is the LCG?
Reduce by online computers that filter out a few
hundred good events per sec
Recorded on disk and magnetic tape at 100-1000
MB/sec 15 PB/year
The LHC Generation of 40 million particle
collisions (events) per second at the center of
each for experiments
Here it is where the GRID environment comes in
4
LCG Service Hierarchy

• Tier-2 100 centres in 40 countries
• Simulation
• End-user analysis batch and interactive

5
Who is who in LCG?
Lab m

• regional group

Uni x
physicsgroup
Uni a
Netherlands
Taiwan
Lab a
UK
Italy
France
Uni n
Tier2
Tier-1
Nordic
Spain
Germany
Canada
USA
Lab b
Lab c
Uni y
Uni b
6
LCG in the World
May 2005 140 Grid sites 34 countries 12000 CPUs 8
PetaBytes
7
How the LCG works?
Ouputs copied to Storage Resources
Sent to the batch system
Connections to UI
SE
Distribution to CPUs
Resources Searching
RB/BDII
Catalogs getting track of the inputs
8
Geant4 in LCG

• ? Electromagnetic and Hadronic physics are
  fundamental features to be properly simulated in
  Geant4, however they are extremely CPU demanding
• ? Number of events and energy depending
  
• 1 event of 1GeV 0.03 sec (2.4GHz machine)
• 1 event of 300 Gev 9-10 sec
• ? Goal during the Software Validation
• Comparison some shower observables between the
  two different Geant4 versions and check
  statistical significant changes
• ? Applications in LCG
• First application last December 2004
• Second application end of June 2005
• ? A total amount of about 3 years of CPU
  time (1GHz machine)
• ? Very small output for the whole
  production 15-20 GB

GRID
GRID
9
Geant4 in LCG

• ? Samplings
• ? 7 simplified detectors
• FeSci, CuSci, PbSci, CuLAr, PbLAr, WLAr, PbW04
• ? 7 different particles (8 in the 2nd
  production)
• e- (2nd production), pi, pi-, k, k-, k0L, p,
  n
• ? 23 different beam energies (GeV)
• 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 20, 30, 40, 50,
  60, 80, 100, 120, 150, 180, 200, 250, 300,
  1000(never achieved)
• ? 5 physics list
• LHEP, QGSP, QGSC, QGSP_BIC, QGSP_BERT

10
Geant4 in LCG

• ? Strategy
• ? First Production
• ? Comparison of 7.0.cand01 vs 6.2.p01
• ? During the event production phase, 5635 had to
  be run for each Geant4 version 11270 jobs to be
  run
• ? Finally the statistical tests were applied to
  each pair of Geant4 version outputs
• ? Second Production
• ? Comparison of 7.0.p01 vs 7.1.cand01
• ? During the event production, 6440 jobs had to
  be run
• ? This time each production job contained the
  production of both Geant4 versions and the
  statistical analysis

11
Geant4 Production in LCG

• ? Stages
• 1. Software installation
• Installation of the Geant4 packages (with all
  the required external additional packages PI,
  AIDA, etc)
• ? Software provided via a tar file
• ? Installation through jobs using specific
  LCG tools
• ? Fundamental request for the sites Shared
  area between WNs and perfectly definition of the
  software installation region
• 2. Events production
• ? Jobs sent by bunches of 1227 (1288 2nd
  production) defined by each physics list
• ? 5000 events in each job were produced
• 3. Analysis
• Statistical tests to perform the comparison
  between the two G4 versions

12
Geant4 Production in LCG

• ? General Characteristics
• ? VO
• ? 1st Production dteam (6 certificates, one
  as dteamsgm)
• ? 2nd Production alice (2certificates, one
  as alicesgm)
• ? Sites and middleware operating system
• ? 1st Production RedHat7.3
• ? 2nd Production Scientific Linux
• ? Resources
• ? 1st Production Own RBBDIIUI lxb2006 at
  CERN
• ? 2nd Production lxplus resources and 2 BDII
  
• ? All output
• ? 1st Production About 30 GB stored at CERN
  (lxn1183)
• ? 2nd Production Comparable quantity stored
  at CERN (lxn1180)
• afs Geant4 area at CERN was set to
  hold the outputs

13
Framework developed for Geant4

• ? Generation of a general framework consisting of
  3 major tools
• ? Tool for general and automatic job
  submission
• ? Tool for events generation in all those
  sites where the software has been installed
• ? Tool for data analysis (not needed during
  the 2nd Production)
• ? First Part Tool for job submission
• Methodology
• ? Copy and registry of the Geant4 package
• ? A file containing the TURL is created and is
  passed to the WN
• ? Follow up of candidates able to admit
  Geant4 jobs
• ? Selection of long queues only
• ? Automatic built of the .jdl files for
  each long queue
• ? Built taking as base those proposed by the
  user joining the name of
• the queue where to submit the job
• ? Software Installation tools are used to
  perform the installation
• ? Submission of these files to each queue

14
Framework developed for Geant4

• ? Software Installation tool
• (Tool submitted in the first step to all sites to
  install the software)
• ? First step
• ? The tar file is copied from the SE at CERN to
  the WN
• ? It is untar and copied to the VO_DTEAM_SW_DIR
  area
• ? Second Step Software Installation tool
• ? Some Geant4 tests are performed to validate
  the installation
• ? If succeeded a tag is published in the
  Information System
• ? Results
• ? The software installation was tried in 63
  sites
• ? 1st Production 28 sites
• ? 2nd Production 35 sites
• ? Main Problems
• ? Sites were having submission problems
• ? Sites did not have defined the VO_DTEAM_SW_DIR
  area or did not
• have shared area among WNs

15
Framework developed for Geant4

• ? Second Step Tool for the Production
• Strategy
• ? Only long queues will be used to run the
  production
• ? All outputs (hbook files) will be stored at
  CERN
• Methodology
• ? Geant4 provides their own code to perform the
  events production
• ? Python Script for each type of particle,
  energy, physics list and calorimeter is created
  by the framework from one template provided by
  Geant4
• ? Generation of one jdl per job containing the
  code provided by Geant4 (the same for all jobs)
  that script generated by the framework and
  changing for each job
• ? Submission of all jdl files to all sites
  containing the Geant4 installation

16
Framework developed for Geant4

• Results (First and Second Production)
• ? A hbook file containing 5000 event is created
  in the case the production succeeded
• ?The name of the file is created by the
  framework containing the type of particles, the
  energy, the physics list and the calorimeter
  within the name (important to perform later the
  comparison)
• ? The hbook file is copied and registered to a
  disk at CERN
• During the 2nd production a tar file containing
  different files should have been created in the
  case the job succeeded. This file was retrieved
  to the afs area delivered for this aim and copied
  and registered to the grid.
• Around 4508 jobs (two physics list for both
  Geant4 versions) were run in lest than 2 weeks in
  28 sites with a efficiency of about 87
• And for the 2nd production the results are
  provided by Alberto Ribon

17
Framework developed for Geant4

• ? Before the analysis stage the outputs should be
  checked (only for the 1st Production)
• ? Dealing with about 5000 outputs is not an easy
  task
• ? Tool able to print in a file all the LFNs in
  the case the efficiency was 100 (reference file)
• ? Output retrieve (only std.out files). If
  succeeded the file will contain the name of the
  LFN
• ? A 2nd tool will check all std.out files
  looking for the succeeded jobs
• - The corresponding GUID and LFN will be stored
  (test file) and compared with the information
  included in the reference file
• ? At this point it was more important for us to
  analyze the successful jobs than to understand
  the cause of the unsuccessful ones (BUT THIS
  PROCEDURE IS WRONG!)

18
Framework developed for Geant4

• ? Third Step Tool for the Analysis (Only for the
  1st Production)
• Methodology
• ? Search of common successful outputs in both
  Geant4 Versions
• ? Each couple of successful outputs are copied
  into a local area and analyzed with their own
  tools
• ? Finally the copy is removed from the local
  area

19
Update of the Framework

• ? This framework covered the Geant4 requirements
  for its first production
• ? Its not useful for larger production
• ? Difficult to deal with the output and
  visualize the results
• ? A new complete tool has been developed for
  large production
• ? Flexible enough to be used for any VO and any
  user application
• ? Most of the improvements mostly relative to
  the outputs handle
• Documentation LCG2 User Guide
• http//grid-deployment.web.cern.ch/grid-deployment
  /cgi-bin/index.cgi?vareis/docs
• Download
• http//goc.grid.sinica.edu.tw/gocwiki/User_tools

20
Update of the Framework

• ? The new framework consists mainly of two tools
• ? Tool to perform the automatic job submission
• ? Tool to retrieve and handle the
  corresponding output
• 1. Automatic job submission
• Overview
• Given an users jdl this tool performs the
  following actions
• ? It lists all sites able to run the jdl
  provided by the user
• ? It creates automatically a jdl file based
  on that provided by the user
• ? It submits the just created jdl
  containing the user application(s)
• Moreover it creates a subdirectory (defined
  by the user) containing a list of the sites where
  the jobs have been submitted, the corresponding
  jdls and the jobs IDs

21
Update of the Framework
Additional Features ? The user can define the
queues where the jobs are submitted. These queues
are checked to see whether it fixes the job
requirements. ? Requested LFN files can be
included. The corresponding TURLs are searched
and included in a file passed in the
InputSandbox to the WN 2. Retrieve and handle of
the outputs ? The 2nd tool checks the status of
the jobs from the job IDs included in the
directory given by the user ? It provides the
following output The job run in
ramses.dcic.ups.es2119/jobmanager-torque-dteam
is in status Scheduled The job run in
grid01.phy.ncu.edu.tw2119/jobmanager-torque-dteam
is in status running The job run in
scaic10.scai.frauhofer.de2119/jobmanager-torque-d
team is in status over The user is queried to
retrieve the output to the destination he has
previously decided
22
Update of the Framework
Additional Features It is possible to
visualize the outputs on the web A html
report is provided showing the files decided by
the user
23
Summary and Conclusions

• ? Satisfactory implementation of the Geant4 code
  in the LCG
• ? (Hopefully this is the beginning of a long
  friendship)
• ? Paper Geant4-LCG submitted to SC05
  Conference in EEUU
• ? The LCG deployment team is quite interested in
  the implementation of the Geant4 code in our own
  tests suites
• ? We hope to collaborate together to help us
  as well
• ? You cannot keep on working borrowing another
  VOs
• ? We have to make you VOGeant4 as soon as
  possible
• ? For the next Geant4 Production this should
  be done
• ? Please come before in each production!
• ? Two weeks is not enough to make a good
  production
• ? We have to understand in all cases the
  failed jobs and this need time