G' Cowan

1
Ideas on Statistics Book Outline
ATLAS Statistics Forum CERN, 21 May, 2007
Glen Cowan Physics Department Royal Holloway,
University of London g.cowan_at_rhul.ac.uk www.pp.rhu
l.ac.uk/cowan
2
ATLAS Statistics Book Outline (DRAFT)
Introduction Overview of basics, probabili
ty, Bayesian vs. Frequentist issues,
notation, terminology, pointers to
resources
3
Statistical tests for selecting events
Optimizing selection Neyman-Pearson,
efficiency, purity, s/sqrt(sb), etc.
Multivariate methods
Considerations concerning what variables to use,
minimizing model uncertainty, validating with
control samples, etc. Overview of impor
tant multivariate methods (linear discriminant,
Neural Networks, SVM, decision trees).
4
Statistical tests establishing
discovery General formalism of goodness-of-fit
test p-value of null hypothesis Including
systematic uncertainties into a measure of
significance Combining measurements
Bayesian hypothesis tests, Bayes factors (?
)
5
Parameter estimation Likelihood function,
properties of estimators (bias,
variance) Overview of important fit methods
Maximum Likelihood, Least Squares. Variance of
estimators information inequality, etc.
Goodness-of-fit in connection with parameter
estimation Robustness, sensitivity of fit to mod
el assumptions, Toy MC test of fit,
etc. Simultaneous fits of several sets of
measurements Including systematic uncertainties
6
Interval estimation (setting limits)
Interval from inversion of a test,
confidence belt construction One-sided, two-
sided, unified (Feldman-Cousins) intervals
Properties of intervals and limits
coverage, mean (median) limit
Multidimensional intervals, confidence
regions Approximate confidence intervals fro
m likelihood function Bayesian intervals
Including systematic uncertainties
7
Systematic uncertainties General considerati
ons, types of systematic uncertainties
Methods for dealing with nuisance
parameters Bayesian methods for systematics
Discussion of recipes for specific types of
systematics, e.g., PDFs, jet energy scale,
theoretical uncertainties, "n results from n
essentially equivalent methods", etc.
Quantifying correlations for systematic
uncertainties
8
Miscellaneous Overview of software tools
Summary of recommendations for statistical
procedures