ATLAS Simulation

1
CHEP04 Interlaken, CH 26 September 1 October
2004
A.Rimoldi, J. Boudreau, D. Costanzo A.
DellAcqua, M. Gallas, A. Nairz, V.Tsulaia
The full detector simulation for the ATLAS
experiment status and outlook
2
outlook

• Simulation data flow
• The ATLAS Simulation Project
• G4ATLAS the Geant4-based simulation for ATLAS
• The ATLAS Simulation from the Data Challenge and
  Physics Validation perspective
• The ATLAS Detector in GEANT4 and its Subdetectors
• the Inner Detector Simulation
• The ATLAS Calorimeters simulation
• The Muon System Simulation
• The ATLAS Testbeam
• The Detector Digitization
• The Simulation Validation Preproduction and DC2
• Memory usage _at_runtime
• Timing for different event samples
• The Data Challenges in ATLAS
• The Physics with DC2 and CTB as a feedback for
  Simulation
• Conclusions

3
ATLAS Simulation data flow
4
The ATLAS Simulation Project

• Present Status
• GEANT3-based simulation was operational for the
  last 10 years, now discontinued (mid 2004)
• It provided a simulation infrastructure used for
  Data Challenges(DC0,DC1), heavy ions, early
  testbeam and design optimization, experiment
  commissioning
• GEANT4-based simulation developed in a full OO
  environment since 2000
• Very detailed and up-to-date in all the previous
  items, in most cases more accurate and performant
  
• used for DC2
• the 2004 combined testbeam, the last before the
  first data-taking
• Heavy ions production
• Ready for early commissioning studies
• A strategy for passage from GEANT3 to GEANT4 was
  successfully launched and followed in ATLAS end
  of last Year
• ATL-SOFT-2003-013 . Strategy for the transition
  from Geant3 to Geant4 in ATLAS. byBarberis, D.
  Polesello, G. Rimoldi, A Geneva CERN, 13 Nov
  2003
• Now the Geant4-based simulation is the main
  simulation engine in ATLAS

5
G4ATLAS the GEANT4-based simulation for ATLAS

• Features
• Completely written in C
• Extensively usage of dynamical loading and action
  on demand
• Completely embedded in the ATLAS ATHENA framework
• Success story in terms of
• open software
• Multi programmers facility
• Results Performance and robustness optimal after
  a short ramp-up
• Started as standalone exercise, then embedded in
  the ATLAS framework, now fully operational for
  experiment and testbeam purposes with the same
  software
• POOL utilized for the I/O
• Functionality
• Most functionality is there. Interactivity is
  provided
• Python scripting replacing the old macro-files
  structure
• Developments
• backward compatibility always provided
• Not the end of the story improvemnet foreseen in
  many fields (background treatment, visualization,
  more interactivity, documentation for end users,
  etc.)

6
The Atlas Simulation in GEANT4 from..

• the Data Challenge viewpoint
• DC0 (since end 2001 tests of event productions
  Geant3 Geant4)
• DC1(Phase I -gtII)
• Geant3 based
• Validation samples (single particle, Et
  scans,Higgs) 740K ev
• Single-particle production 30 million ev
• Minimum-bias production 1.5 million ev
• QCD di-jet production 5.2 million ev
• Physics events requested by HLT groups 4.4
  million ev
• Pile-up
• Data samples requests from end-user community
• DC2 and following
• GEANT4 based
• large scale physics analysis, tests on computing
  model, test calibrations and alignment procedures
• 12 millions fully simulated events
• And a grand total of 1 job crash !
• Distributed production
• 1M Z-gtee events in 10K jobs and no failures
  (_at_NorduGrid)
• the Physics Validation viewpoint

7
The ATLAS detector in Geant4

• Four main subdetectors
• Inner detector - momentum measurement of charged
  particles, electron ID
• high precision silicon trackers
• straw tracker with TR capability
• Calorimeters - measurement of particle energies
• EM LAr calorimeters (barrel endcap)
• Hadron Lar calorimeters (endcap)
• Scintillating Tile hadron calorimeter
• Muon spectrometer - muon identification and
  measurement
• High precision Drift Tubes for tracking, RPCs and
  TGCs for triggering
• Magnet system - bending of charged particles for
  momentum measurements
• 5.2M volumes objects(G3 27M)
• 110K volume types (G3 23K)

8
The Inner Detector

• Geometry
• Three subdetectors components
• Pixel
• SCT
• TRT
• Final / initial layout available, preliminary
  validation on hits content done
• Still to do
• Allow global movements of the Pixel
• Increase the level of details
• Detector response
• Tuned on test beam results
• Home-grown TR model
• Digitization
• Adapt hit reading for pile-up
• Introduce the concept of time in the digitization
• Noise for TRT

9
The ATLAS Calorimeters Simulation

• 4 subsystems
• Electromagnetic barrel (EMB)
• Hadronic end-cap (HEC)
• Forward calorimeter (FCal)
• Hadronic calorimeter (Tile)
• Heavy tests/investigations for optimizing
• the physics in Geant4
• the geometry for reducing memory consumption and
  cpu time
• Parameterization studies ongoing
• Main software infrastructure issues
• the detector description
• no more hand-coded numbers, full GeoModel
  version available (library of geometrical
  primitives for describing detector geometries)
• -gt V.Tsulaia talk 279, tomorrow
• the versioning of the database constants
• LAr has already been switched to Oracle
• the inclusion of calibration hits
• do a careful accounting of where the energy goes
  in ATLAS

10
The Muon System Simulation

• System composed by
• Four main subdetectors
• 2 precision chambers tracking detectors
• MDT, CSC
• 2 trigger chambers detectors
• RPC,TGC
• Interleaved with the toroids structure
• Feets and rails
• The outermost detector of ATLAS
• Services and cables passing through
• Pileup cavern background
• Functionality for handling pileup in place
• digitization time window for all technologies
• Current DC2 production no cavern background yet,
  only minimum-bias
• Full background as in DC1 expected soon
• CombinedTestBeam (Muon side)
• Robustness of sim-digi chain demonstrated through
  tons of events produced
• Comparisons with real data for all technologies

11
The ATLAS Testbeam

• The 2004 CTB case
• All the components for a complete ATLAS sector
  are being tested together on a beam line in
  Summer 2004 (Combined TestBeam Setup)
• _at_different beam energies
• Magnetic field (2)
• Ancillary detectors
• Customized beam profile at generation
• Deep comparisons with real data sample in each
  subdetector prototype
• Fine tuning
• Full chain Simulation-Digitization-Reconstruction
  done!.
• In production mode.
• Same software as for the full experiment

12
The Detector Digitization

• The digitization procedure is disentangled from
  the simulation process proper and it can be
  started from pregenerated hits or in a full chain
• Fully functional and "leak free" since months
• Each subdetector loaded on demand
• Digitization of GEANT4 hits done
• Hits I/O with Pool for all subdetectors
• Expect extensive Validation work from the CTB by
  comparing with data
• All assumptions (resolutions, smearing, ) to be
  revisited
• Pile up
• Pile-up stays in 1Gbyte of memory, takes
  5mn/event (1GHz) and needs 10Mbyte/event of disk
• Pythia used for the min-bias pile up events
• A set of 500K min bias events with the atlas
  tunings is used
• To do
• Optimize I/O use vs. memory
• Occupancy studies need to be done
• MC truth navigation, not supported for pile-up
• Lot of work still needed. Both for Validation and
  new Development

13
The Simulation Validation Preproduction and DC2

• preliminary tests started at end 2003
• comparisons with Geant3 using common event
  samples and about the same geometry.
• Hits and digits for all the ATLAS subdetectors
  generated
• jobs run in parallel using the LSF batch facility
  and Castor facility (at CERN and outside)
• we measured at different event/run phases the
  local peformance and memory usage
• Generated samples
• Single particle vs. E
• SUSY events
• H-gt4 leptons, Z-gt2leptons(e, mu,tau)
• di-jet
• minimum bias
• Initial failure rate of 10 for single particle
  jobs (30 in physics events), corrected patiently
  (geometry problems, G4 physics problems)
• Final failure rate is approximately 0 apart from
  AFS or Castor problems.
• All jobs go straightforward to the end

14
Watching the application _at_runtime
Configuration _at_ Process Size _at_runtime (MB)
G4 initialisation (application alone) 50 MB
building ATLAS envelopes 0.9 MB
building ID Geometry SensitiveDetector 2.2 MB 0.3 MB
building LArCalo 83.7 MB
building Tile Geometry SensitiveDetector 0.6 MB 0.3 MB
building Muon Geometry SensitiveDetector 8.0 MB 0.2 MB
geometry optimisation 30 MB
magnetic field 12 MB
loading of full (default) Physics Lists 49 MB
External Dependencies External Dependencies
reading events from ROOT files 27 MB
declaring POOL in jobOptions (hit persistification) 40 MB
LAr hit calibration 100 MB
using ORACLE database (under study now) 100 MB
ID GeoModel _at_ initialisation (overhead) 18 MB

• Inspections _at_runtime allow to
• control the memory consumption
• control the possible memory leaks during data
  production
• evaluate pros / cons when a new feature is
  implemented
• They are possible everywhere in the production
  flow, particulary useful at
• Begin of Run
• Begin of Event
• Begin of Step
• And at EOR, EOE, EOS

15
G4 Timing single particle and full events
Heavy ions (Hijing) 3 full events produced
16
The Physics with DC2 CTB as a feedback to
Simulation

• Our goals
• Get physics community familiar with the new
  software, new persistency, new analysis tools
• Use large produced samples to understand
  performance and tune new simulation
• Test algorithms on CTB data
• Understand limitations of simulation
• Understand key issues for reconstruction
  algorithms
• Tune simulation parameters
• Get to the end of the year with large
  well-understood samples of simulated data, stable
  and tested software chain
• Full simulation analyses (signal background)
  for initial detector setup on key physics
  channels
• Inject additional realism into simulation studies
• Waiting for feedback from our users community

17
Conclusions

• The Simulation Geant4-based was successfully
  tested and it has by now replaced the
  Geant3-based one
• We extensively measured the performance and
  robustness of the new simulation with great
  success
• We can use Parameterizations for further
  improving the simulation performance
• Geant4 is the main engine for the simulation in
  ATLAS

18
Thanks

• To all the core developers
• For the robust, versatile and complete code
  provided
• To the subdetector people
• For their prompt implementation of any new
  functionality provided
• To the GEANT4 collaboration
• for their help in transforming this exercise into
  a success