The CMS Simulation Validation Suite

1
The CMS Simulation Validation Suite

• V. Daniel Elvira (Fermilab)
• for the CMS Collaboration

2
Simulation Software in CMS
Generation MC truth information from particle
gun or physics generator about vertices and
particles. Stored in HepMC format. Detector
Simulation Hit objects with timing, position,
energy loss information. Based on the Geant4 tool
kit. Digitization Constructs Digi objects
which include realistic modeling of electronic
signal. Reconstruction Physics Objects
vertices, photons, e, m, jets,
3
Physics Software Validation

• Elements to validate
• Geometry description magnetic field map
• Physics of EM and Had showers in Geant4 - hits
• Digitization Model - digis
• Reconstruction Algorithms physics objects

For absolute validation Use visualization tools,
TB experiments, reference plots
The Simulation Validation Suite (SVS) validates
in an automated way each new release of the CMS
simulation software, comparing values of
quantities related to geometry, field, hits with
reference values from a previously (absolutely)
validated version.
Expansion to Physics Software Validation Suite
including digis, reco in progress
4
CMS Detector Systems
SVS modular structure follows the detector
sub-systems
Solenoid Magnet 4 Tesla Field
Muon System
Electromagnetic Calorimeter (Ecal)
Hadronic Calorimeter (Hcal)
Silicon Tracker
22 m long 15 m in diameter
More than 1 Million Geometrical volumes
5
SVS General Description
OVAL testing tool created by CMS to detect
changes in software behavior.
Used as the SVS integration tool. It executes
scripts and shell commands to control the suite
execution and perform comparison tests.
Pre-generated samples single particles, pp
physics (Pool files, ROOT browsable, HepMC
format)
SVS software sub-systems (dedicated
simulation packages)
Validation branch (Pool files, ROOT browsable)
SimG4TrackerValidation SimG4EcalValidation SimG4Hc
alValidation SimG4MuonValidation SimG4GeomValidati
on SimG4FieldValidation SimG4GlobalValidation
OVAL
Configuration file commands, tolerance values
On the fly analysis basic G4 objects processed
into information to construct validation
quantities.
6
SVS General Description (II)
Validation branch (Pool files, ROOT browsable)
ROOT Analysis Macros
OVAL

• Construct Validation
• Quantities
• Create/fill histograms
• Read reference files
• Perform Validation Tests
• c2 or Kolmogorov-Smirnov

Reference Histograms from previous version (ROOT
file)
Configuration file commands, tolerance values
(LCG PI Statistics Testing toolkit, or ROOT)
One or more tests per sub-detector low/high
level quantities, different sub-detector
components.
List of differences for quantities not passing
the tests (ASCII files)
Results are reviewed by system experts approve
release, or investigate problems
7
Tracker Validation
Tracking system Silicon Strip Tracker -
Tracker Inner Barrel (TIB), Tracker Outer
Barrel (TOB),
Tracker Inner Disks (TID)
and Tracker End Cap (TEC)
Pixel Detector - Pixel Barrel and Pixel End Cap.
Sample Single muons, electrons or pions with pT
15 GeV in 12 bins of in the range -3lthlt3
(1,500 events)
Identical Distributions
Reference Distribution
Validation Quantities Energy deposition Distribut
ion of track entry and exit points Number of
hits Time of flight,, etc
Current Distribution
8
Ecal Validation
Electromagnetic Calorimeters Ecal Barrel
Ecal Endcap (crystals)
Preshower
(Si/lead)
Low Level Test Single 30 GeV photons (2,000
events)

• h vs f hit occupancy in crystals
• E1, E2x2, E3x3, E4x4, E5x5 depositions
• E1/E4, E4/E9, , E9/E25 ratios
• Percent of E in Barrel, Endcap, Preshower
• Longitudinal shower development

High Level Test Single 10GeV, 20GeV, 30GeV,
40GeV, 50GeV (2,000 events)

• E25 resolution vs the incident energy
• Longitudinal shower development vs energy
• Coefficient of Lead absorption vs incident
• energy (preshower)

9
Ecal Validation (II)
10
Hcal Validation
Hadronic Calorimeters Hcal Barrel Hcal
Endcap (copper/scint.)
Hcal Forward (steel/quartz
fibers)
Single 50 GeV pions (1,000 events)

• E depositions in each HB layer
• Time dist. of Hit energy in 7x7 tower matrix
• Number of hits in Ecal and Hcal
• Energy in 1x1, 3x3, 5x5 tower matrices

• Total energy in Hcal
• Total energy in long fibers (HF)
• Total energy in short fibers (HF)

Time distribution in 7x7 tower matrix
Layer 0
11
Muon Validation
Muon System Drift Tubes (DT) in central
region. Cathode
Strip Chambers (CSC) in forward region.
Resistive Plate Chambers
(RPC) in both for trigger.
Single 100 GeV muons (1,000 events)
E lost by m in full CMS detector For 100 GeV
m, ltEgt8 GeV
For each muon Energy lost Deviation in
position Deviation in angle (deg) Number of
tracking steps
For each type of muon process (ionization,
bremsstrahlung, ee- production, muon nuclear
interaction, decay and capture) Energy of
secondary particles Angle of secondary track
with respect to primary muon track
12
Geometry Validation
Computes the number of volumes and materials and
the total number of radiation lengths through the
CMS detector.
Sample
1,000 m of E10 TeV and random h, f ( neutral
geantinos since physics/field off)
Validation Quantities
The geometry summary number of different
materials, number logical volumes, physical
volumes. The number of radiation lengths after
traversing the full detector.
Material Budget (ROOT tree, histos, or ascii file)
For each Geant4 step of each muon Accumulated
track length, volume name, volume copy number,
accumulated material budget (Number of radiation
lengths), material radiation length. At the end
of track Accumulated material budget.
13
Field Validation
Checks the tracking in the CMS magnetic field.
The test compares the deviation at the end of the
track in position and direction.
Sample
400 single muon events in four groups of 100
events with different energies 1, 10, 100, 1000
GeV. The muons behaves as charged geantinos
(physics is off).
Validation Quantities
Change in track angle, momentum, kinetic energy.
14
Global Validation
Sub-system packages dedicated validation tests
for individual sub-systems. Geometry sub-set,
field off. Global package validates entire
detector with real field, using pp physics
samples. For example, min-bias.

• Monte Carlo
• Number of vertices from Geant4
• Number of tracks from Geant4
• Number generated particles
• Position (x,y,z) of each Geant4 vertex
• pT of each Geant4 track
• Energy of each Geant4 track

2. Electromagnetic Calorimeter Number of Ecal
hits Energy of each Ecal hit Time of flight for
each Ecal hit Global h of each Ecal hit Global f
of each Ecal hit Number of preshower Hits Energy
of each preshower hit Time of flight for each
preshower hit Global h of each preshower
hit Global f of each preshower hit
15
Global Validation (II)
3. Hadronic Calorimeter Number of hits Energy of
each hit Time of flight for each hit Global h of
each hit Global f of each hit
4. Tracker Number of Pixel hits Global h of each
Pixel hit Global f of each Pixel hit Time of
flight of forward Pixel hits Time of flight of
barrel Pixel hits Global R of barrel Pixel
hits Global Z of forward Pixel hits Number of
Silicon hits Global h of each Silicon hit Global
f of each Silicon hit Time of flight of forward
Silicon hits Time of flight of barrel Silicon
hits Global R of barrel Silicon hits Global Z of
forward Silicon hits
5. Muon Number of hits Global h of hits Global f
of hits Time of flight for DT hits Global R of DT
hits Time of flight for CSC hits Global Z of CSC
hits Time of flight for RPC barrel hits Global R
of RPC barrel hits Time of flight for RPC forward
hits Global Z of RPC barrel hits
16
Summary Outlook
CMS simulation validation suite for hit derived
quantities is operational being ported to the
CMSSW framework

• Iterate on validation quantities
• Optimize sample types, size, tune tolerance
  values

In progress

• Expand Physics Validation Suite to include Digis
  (pulse
• shape, pedestals, gains)

In the future

• Incorporate reconstructed physics objects jets,
  e, g, m