Geant4 Physics Validation (mostly electromagnetic, but also hadronic

1
Geant4 Physics Validation(mostly
electromagnetic, but also hadronic)

• K. Amako, S. Guatelli, V. Ivanchenko, M. Maire,
  B. Mascialino, K. Murakami, P. Nieminen, L.
  Pandola, S. Parlati, M.G. Pia, T. Sasaki, L.
  Urban
• et al.

Geant4 Space User Workshop Leuven, 5-7 October
2005
2
Geant4 Physics Models

• Ample variety of physics models in the Geant4
  Toolkit
• complementary and alternative
• Electromagnetic physics
• Standard, LowEnergy, Muon, Optical
• Hadronic physics
• data-driven, parameterised and theory-driven
  models
• Geant4 Physics Book
• on-going project to document the performance of
  Geant4 physics against experimental data and in
  relevant experimental application domains

3
Validation process

• Geant4 test process
• Physics packages are subject to unit and system
  testing
• Verification, validation of single
  processes/models performed by Working Groups
• Validation process
• systematic cover all models of a given process
• comparison to experimental data and established
  reference databases
• rigorous software process to guarantee quality
  and reliability
• statistical analysis quantitative mathematical
  evaluation
• Goals
• evaluate quantitatively the accuracy Geant4
  physics models
• document their respective strength
• provide guidance to users to select the models to
  use in their applications

4
G.A.P Cirrone, S. Donadio, S. Guatelli, A.
Mantero, B. Mascialino, S. Parlati, M.G. Pia, A.
Pfeiffer, A. Ribon, P. Viarengo A
Goodness-of-Fit Statistical Toolkit IEEE-
Transactions on Nuclear Science (2004), 51 (5)
2056-2063
Partly funded by ESA (SEPTIMESS Project)
5
GoF algorithms (currently implemented)

• Algorithms for binned distributions
• Anderson-Darling test
• Chi-squared test
• Fisz-Cramer-von Mises test
• Tiku test (Cramer-von Mises test in chi-squared
  approximation)
• Algorithms for unbinned distributions
• Anderson-Darling test
• Cramer-von Mises test
• Goodman test (Kolmogorov-Smirnov test in
  chi-squared approximation)
• Kolmogorov-Smirnov test
• Kuiper test
• Tiku test (Cramer-von Mises test in chi-squared
  approximation)
• In progress
• Watson test
• Girone Test
• Weighted Kolmogorov-Smirnov test
• Weighted Cramer-von Mises test
• The most complete GoF software system on the
  market
• even among commercial/professional statistics
  software products)

6
Overview of recent validation activities

• Geant4 Physics Book Electromagnetic Volume
• comparison against the NIST databases
• K. Amako, S. Guatelli, V. N. Ivanchenko, M.
  Maire, B. Mascialino, K. Murakami, P. Nieminen,
  L. Pandola, S. Parlati, M. G. Pia, M.
  Piergentili, T. Sasaki, L. UrbanComparison of
  Geant4 electromagnetic physics models against the
  NIST reference dataIEEE Trans. Nucl. Sci., Vol.
  52, Issue 4, Aug. 2005, 910-918
• Current Physics Book projects (preliminary
  results)
• Bremsstrahlung final state
• Atomic relaxation and PIXE
• Bragg peak
• Radioactivity from rocks and sands
• Other Geant4 validation activities
• LCG Simulation Validation Project focus on
  hadronic physics
• see A. Ribonss talk at EPS-HEP, Lisbon, July
  2005
• Validation of specific physics models done by
  each Geant4 Working Groups

7
NIST Test

• Photon Mass Attenuation Coefficient
• Photon Partial Interaction Coefficient
• related to the cross section of a specific photon
  interaction process
• Electron CSDA range and Stopping Power
  
• Proton CSDA range and Stopping Power
  
• a CSDA range and Stopping Power

Elements Be, Al, Si, Fe, Ge, Ag, Cs, Au, Pb,
U (span the periodic element table)
Geant4 models electrons and photons Standard Low
Energy EEDL/EPDL Low Energy Penelope
Energy range photon 1 keV 100 GeV electron
10 keV 1 GeV proton 1 keV 10 GeV a
1 keV 1 GeV
Geant4 models protons and a Standard Low Energy
ICRU49 Low Energy Ziegler 1977 Low Energy Ziegler
1985 Low Energy Ziegler 2000 (Low Energy free
electron gas parameterisations Bethe-Bloch)
Simulation configuration reproducing NIST
conditions (ionisation potential, fluctuations,
production of secondaries etc.)
8
Statistical analysis
Goodness-of-Fit test (Statistical Toolkit)
H0 Geant4 simulation NIST data H1 Geant4
simulation ? NIST data
Alternative hypotheses under test
Geant4 simulation results ReferenceData
Distance between Geant4 simulation and NIST
reference data
GoF test (?2 test)
Test result p-value
The p-value represents the probability that the
test statistics has a value at least as extreme
as the one observed, assuming the null hypothesis
is true 0 p 1
p lt 0.05 Geant4 simulation and NIST data differ
significantly p gt 0.05 Geant4 simulation and
NIST data do not differ significantly
9
Photon mass attenuation coefficient

• Geant4 models
• Standard
• Low Energy EPDL
• Low Energy Penelope
• Reference data NIST - XCOM

Experimental set-up
Results All Geant4 models compatible with
NIST Best agreement Geant4 LowE models
p-value stability study
10
Compton interaction coefficient (cross section)

• Geant4 models
• Standard
• Low Energy EPDL
• Low Energy Penelope
• Reference data NIST - XCOM

Results All Geant4 models compatible with
NIST Best agreement Geant4 LowE-EPDL
11
Photoelectric interaction coefficient (cross
section)

• Geant4 models
• Standard
• Low Energy EPDL
• Low Energy Penelope
• Reference data NIST - XCOM

Results All Geant4 models compatible with
NIST Best agreement Geant4 LowE models
12
Pair production interaction coefficient (cross
section)

• Geant4 models
• Standard
• Low Energy EPDL
• Low Energy Penelope
• Reference data NIST - XCOM

Results All Geant4 models compatible with NIST
and equivalent
13
Rayleigh interaction coefficient (cross section)
Results The Geant4 Low Energy models look in
disagreement with the reference data for some
materials

• Geant4 models
• Low Energy EPDL
• Low Energy Penelope
• (no standard Rayleigh process)
• Reference data NIST - XCOM

EPDL XCOM
Penelope XCOM
Rayleigh interaction coefficient in Be
Be 0.99 1
Al 0.32 lt0.05
Si 0.77 lt0.05
Fe 1 lt0.05
Ge lt0.05 0.39
Ag 0.36 0.08
Cs lt0.05 lt0.05
Au lt0.05 lt0.05
Pb lt0.05 lt0.05
U lt0.05 lt0.05
14
Rayleigh interaction coefficient
The disagreement is evident between 1 keV and 1
MeV photon energies
For what concerns the Geant4 Low Energy EPDL
model, the effect observed derives from an
intrinsic inconsistency between Rayleigh cross
section data in NIST-XCOM and the cross sections
of EPDL97, on which the model is based
Differences between EPDL97 and NIST-XCOM have
already been highlighted in a paper by Zaidi,
which recommends the Livermore photon and
electron data libraries as the most up-to-date
and accurate databases available for Monte Carlo
modeling.
Zaidi H., 2000, Comparative evaluation of photon
cross section libraries for materials of interest
in PET Monte Carlo simulation IEEE Transaction on
Nuclear Science 47 2722-35
15
Experimental set-up
Electron Stopping Power
Electrons are generated with random direction at
the center of the box and stop inside the box

• Geant4 models
• Standard
• Low Energy EEDL
• Low Energy Penelope
• Reference data NIST ESTAR (ICRU 37)

Maximum step allowed in tracking particles was
set about1/10 of the expected range value, to
ensure the accuracy of the calculation
Results All Geant4 models compatible with
NIST and equivalent
16
CSDA particle range without energy loss
fluctuations and multiple scattering
Electron CSDA Range

• Geant4 models
• Standard
• Low Energy EEDL
• Low Energy Penelope
• Reference data NIST ESTAR (ICRU 37)

Results All Geant4 models compatible with
NIST and equivalent
p-value stability study
H0 REJECTION AREA
17
Proton stopping power - range
Stopping power p-value stability study
H0 REJECTION AREA
CSDA range p-value stability study
Results Ziegler parameterisations are as
authoritative as ICRU 49 ones Comparison rather
than validation
H0 REJECTION AREA
18
a stopping power and range
The complex physics modeling of ion interactions
in the low energy range is addressed by the
Geant4 Low Energy package and it represented one
of the main motivations for developing this
package
The complex physics modeling of ion interactions
in the low energy range is addressed by the
Geant4 Low Energy package and it represented one
of the main motivations for the developing of
this package.
19
Bremsstrahlung
3 sets of models Standard G4eBremsstrahlung Low
Energy EEDL G4LowEnergyBremsstrahlung Low Energy
Penelope G4PenelopeBremsstrahlung
3 angular distributions Tsai, 2BS, 2BN
Penelope Standard Low Energy EEDL (default)
Penelope LowE-EEDL TSAI (def) LOWE-EEDL
2BS LOWE-EEDL 2BN
Angle (deg)
Angle (deg)
Angular distribution of photons is strongly
model-dependent
20
Reference data
Transmitted energy spectrum at two different
emission angles for four materials (Al, Pb, W,
Ag)
Absolute yields are reported ( photons/primary),
though with an odd normalization
The absolute Bremsstrahlung cross section can be
tested
R. Ambrose et al., Nucl. Instr. Meth. B 56/57
(1991) 327
21
Relative comparison...
Preliminary
Low E EEDL - TSAI
LowE-Penelope
Intensity/Z (eV/sr keV)
Intensity/Z (eV/sr keV)
Photon energy (keV)
Photon energy (keV)
Relative comparison (45 degree direction) Shapes
of the spectra are in good agreement Work in
progress, will be published
22
Proton Bragg Peak
Reference data from CATANA (INFN-LNS
Hadrontherapy Group)

• Geant4 models electromagnetic
• Standard
• Low Energy ICRU 49
• Low Energy Ziegler 1977
• Low Energy Ziegler 1985
• Low Energy Ziegler 2000
• Geant4 models hadronic
• Precompound default de-excitation
• Precompound GEM evaporation
• with/without Fermi Break-up
• Binary Cascade
• (including Precompound de-excitation)
• Bertini Cascade
• Bertini Cascade Bertini elastic scattering
  (when available)
• Parameterised
• Geant4 educated guess Medical Dosimetry Physics
  List

Systematic test in progress Lot of
work Preliminary results
23
EM only Standard
Preliminary
24
EM only ICRU49
ENTIRE PEAK Exp G4
S 2.89 2.43
T 3.26 3.83
GoF test CVM-AD CVM-AD
ENTIRE PEAK Exp G4
S 2.89 2.43
T 3.26 3.83
GoF test CVM-AD CVM-AD
Preliminary
LEFT TAIL xlt30mm Exp G4
S 9.77 11.89
T 2.66 3.16
GoF test CVM-AD CVM-AD
RIGHT TAIL Xgt30mm Exp G4
S 3.89 12.24
T 1.03 1.00
GoF test KS-AD KS-AD
25
EM only ICRU49 GoF results
ALL (N1149 N267) CVM AD
Test statistics 0.112938 0.853737
p-value 0.525095 0.443831
LEFT TAIL xlt30mm (N1140 N261) CVM AD
Test statistics 0.0701584 0.645422
p-value 0.750593 0.606120
Preliminary
RIGHT TAIL Xgt30mm (N19 N26) KS AD
Test statistics 0.333333 0.816534
p-value 0.724871 0.469251
26
LowE precompound default
ALL Exp G4
S 2.89
T 3.26
GoF test CVM-AD CVM-AD
Preliminary
LEFT TAIL xlt30mm Exp G4
S 9.77
T 2.66
GoF test AD AD
RIGHT TAIL Xgt30mm Exp G4
S 3.89
T 1.03
GoF test KS-AD KS-AD
27
ICRU49 precompound GoF results
Preliminary
ENTIRE PEAK (N1149 N266) CVM AD
Test statistics 0.06 0.499375
p-value 0.79 0.747452
ICRU 49 only
0.525095 0.443831
LEFT TAIL xlt30mm (N1140 N260) CVM AD
Test statistics 0.03 0.232255
p-value 0.97 0.978972
0.750593 0.606120
RIGHT TAIL Xgt30mm (N19 N26) KS AD
Test statistics 0.33 0.901787
p-value 0.73 0.413129
0.724871 0.469251
28
Nuclear de-excitation alternative models
Preliminary
Work in progress, more to come
29
Radioactive spectrum
Studies of environmental radioactivity from rocks
and sands at the Gran Sasso Laboratory
Geant4 (LowE EM) can the results of a calibration
with a 60Co source (in the presence of the
sample) reproduce very well

• Anderson-Darling test (for binned data)
• Lower E peak
• A2 0.45
• p-value 0.80
• Higher E peak
• A2 1.05
• p-value 0.33
• Both peaks
• A2 0.80
• p-value 0.48

Lower part of the histogram is not meaningful
30
(No Transcript)
31
Backscattering coefficient E100keV
Angle of incidence (with respect to the normal to
the sample surface) 0
32
and much more

• No time to present all the on-going activities
• atomic relaxation
• LCG Simulation Validation project
• validation for specific use cases (medical
  dosimetry, radiation damage to components,
  underground experiments etc.)
• Physics validation is a large component of Geant4
  Collaborations activities
• Geant4 Physics Validation Workshop, Genova, July
  2005
• http//www.ge.infn.it/geant4/events/july2005/

33
Geant4 validation is not an easy job
experimental data often exhibit large differences!
34
Conclusion

• More results available, no time to show them all
• Systematic, quantitative validation of Geant4
  physics in progress
• all available models
• rigorous statistical analysis
• A lot of work!
• first paper published
• several on-going projects
• limited resources available
• The validation work provides valauable feedback
  for the improvement of Geant4 physics models

35
Communication Publications

• Feedback from Geant4 users is a very helpful
  contribution to Geant4 validation
• if you have a problem with Geant4 physics, please
  tell us
• if you get nice results, please tell us too
• IEEE Trans. Nucl. Sci.
• major scientific journal about nuclear
  technologies and instrumentation
• many Geant4-related papers published or currently
  in the review process
• M.G. Pia Associate Editor (software -
  Instrumentation)
• please consider publishing your results
  concerning Geant4 applications