Bayesian Methods

1
Bayesian Methods
2
Introduction of course

• Introduction myself
• Introduction students name, from, enrolled
  status, reason just interested/particular
  research question ...
• Sign up sheet
• Syllabus, course logistics

3
What is statistics

• The study about uncertainty
• Statistics is the exploration of a parameter
  \theta in light of data X
• Dennis Lindley

4
Contrast between classical and Bayesian statistics

• classical

• Bayesian

What is statistics? Whats probability? Data
X Parameter ? Confidence Interval
5
What is statistics

• Classical
• Mathematical statistics ... deals with problems
  relating to the performance characteristics of
  rules of inductive behavior based on random
  experiments
• --Jerzy Neyman

• Bayesian
• Statistics is the study of uncertainty, most
  commonly, the exploration of a parameter ? in
  light of data X
• --Dennis Lindley

6
What is probability

• Classical
• frequency of a certain event when repeating a
  random experiment infinite times
• Examples
• 1. probability of heads in a coin toss
  experiment defined
• 2. probability that it will snow 1/22/2009 not
  defined.
• 3. probability of me getting cancer by 70 if
  smoking 1 pack cig a day not defined

• Bayesian
• the quantification of uncertainty de Finetti
• this uncertainty can be equated with personal
  belief of a quantity
• Examples
• 1. belief (can be updated by observation)
• 2. belief (can be refined by modeling)
• 3. belief (as above)

7
Data and Parameter

• Classical
• Parameter is fixed
• Data is random

• Bayesian
• Data is given (fixed in operational sense)
• Parameter has uncertainty (random)

• Goal of both draw conclusion/inference for ?

8
Data and Parameter (cont.)

• Classical
• Parameter ? is fixed
• Data X is random

• Bayesian
• Inference should be the same if data are the
  same likelihood principle
• We can talk about distribution of ?, to express
  uncertainty/belief about ?, whereas classical
  stats cannot.

9
Confidence/credible Interval for ?

• (CI is random) If samples of the same size are
  drawn repeatedly from a population specified by
  ?, and a confidence interval is calculated from
  each sample for ?, then 95 of these intervals
  should contain ?.

• Probability of 95 that the credible interval
  contains ?.

10

• Bayesian approach respects likelihood principle
  (only based on data X)
• Classical procedures may violate it. e.g.
  p-valuePr(X or more extremeH0)
• Optimal rules in the classical sense turn out to
  be Bayes rules
• The foundation of statistics can only be
  established within the Bayesian framework.

11
Bayesians use priors

• Prior Non-experimental knowledge
• Examples from Bergers book
• Music expert, distinguish mozart from haydn
• A lady tasting tea tea poured into milk or vice
  versa.
• A man predicts coin toss.
• Each of the 10 times, they got it right.
• Inference may differ due to different prior
  beliefs.

12
Criticisms of Bayesianism

• Use of prior not objective different people
  come to different conclusions this is not
  scientific
• Absolute objectivity does not exist. Appearing
  to be objective sweeps issues under the carpet.
• This provides a formal way of incorporating
  subjective belief
• To be objective use noninformative priors
• Evidence-based when data accumulate, all
  rational individuals will come to the same
  conclusion

13
Bayesian Machinery

• data
• prior belief ------------? posterior belief
• likelihood
• prior distribution ------------? posterior
• L(?)P(X?)
• p(?) --------------------? p(?X)

14
Bayesian Machinery

• Bayes rule

or
15
Example

• ? probability of baby girls in placenta previa
  birth
• One prior belief on ? beta(3,3)

16
Observe 2 girls out of 2 births
prior (beta)
likelihood (binomial)
posterior
posterior mean
posterior 95 credible interval (0.29, 0.90)
17
Classical approach

• Point estimation

confidence interval (?,?)

• Hypothesis testing
• H0 ?0.5
• p-valueP(observing 2 or more female births out
  of 2?0.5)0.25 ? fail to reject