Introduction to Bayesian statistics

1
Introduction to Bayesian statistics

• Three approaches to Probability
• Axiomatic
• Probability by definition and properties
• Relative Frequency
• Repeated trials
• Degree of belief (subjective)
• Personal measure of uncertainty
• Problems
• The chance that a meteor strikes earth is 1
• The probability of rain today is 30
• The chance of getting an A on the exam is 50

2
Problems of statistical inference

• Ho ?1 versus Ha ?gt1
• Classical approach
• P-value P(Data ?1)
• P-value is NOT P(Null hypothesis is true)
• Confidence interval a, b What does it mean?
• But scientist wants to know
• P(?1 Data)
• P(Ho is true) ?
• Problem
• ? not random

3
Bayesian statistics

• Fundamental change in philosophy
• T assumed to be a random variable
• Allows us to assign a probability distribution
  for ? based on prior information
• 95 confidence interval 1.34 lt ? lt 2.97 means
  what we want it to mean
• P(1.34 lt ? lt 2.97) 95
• P-values mean what we want them to mean P(Null
  hypothesis is false)

4
Estimating P(Heads) for a biased coin

• Parameter p
• Data 0, 0, 0, 1, 0, 1, 0, 0, 1, 0
• p 3/10 0.3
• But what if we believe
• coin is biased in favor
• of low probabilities?
• How to incorporate prior beliefs into model
• Well see that p-hat .22

5
Bayes Theorem
6
Example

• Population has 10 liars
• Lie Detector gets it right 90 of the time.
• Let A Actual Liar,
• Let R Lie Detector reports you are Liar
• Lie Detector reports suspect is a liar. What is
  probability that suspect actually is a liar?

7
More general form of Bayes Theorem
8
Example

• Three urns
• Urn A 1 red, 1 blue Urn B 2 reds, 1 blue
  Urn C 2 reds, 3 blues
• Roll a fair die. If its 1, pick Urn A. If 2 or
  3, pick Urn B. If 4, 5, 6, pick Urn C. Then
  choose one ball.
• A ball was chosen and its red. Whats the
  probability it came from Urn C?

9
Bayes Theorem for Statistics

• Let ? represent parameter(s)
• Let X represent data
• Left-hand side is a function of ?
• Denominator on right-hand side does not depend on
  ?
• Posterior distribution Likelihood x Prior
  distribution
• Posterior distn Constant x Likelihood x Prior
  distn
• Equation can be understood at the level of
  densities
• Goal Explore the posterior distribution of ?

10
A simple estimation example

• Biased coin estimation P(Heads) p ?
• 0-1 i.i.d. Bernoulli(p)
  trials
• Let be the number of heads in n trials
• Likelihood is
• For prior distribution use uninformative prior
• Uniform distribution on (0,1) f(p) 1
• So posterior distribution is proportional to
• f(Xp)f(p)
• f(pX)

11
Coin estimation (contd)

• Posterior density of the form f(p)Cpx(1-p)n-x
• Beta distribution Parameters x1 and n-x1
• http//mathworld.wolfram.com/BetaDistribution.html
  
• Data 0, 0, 1, 0, 0, 0, 0, 1, 0, 1
• n10 and x3
• Posterior distn is Beta(31,71) Beta(4,8)

12
Coin estimation (contd)

• Posterior distn Beta(4,8)
• Mean 0.33
• Mode 0.30
• Median 0.3238
• qbeta(.025,4,8),
• qbeta(.975,4,8)
• .11, .61 gives 95
• credible interval for p
• P(.11 lt p lt .61X) .95

13
Prior distribution

• Choice of beta distribution for prior

14

• Posterior Likelihood x Prior
• px(1-p)n-x
  pa1(1-p)b1
• pxa1(1-p)n-xb1
• Posterior distribution is Beta(xa, n-xb)

15
Prior distributions

• Posterior summaries
• Mean (xa)/(nab)
• Mode (xa-1)/(nab-2)
• Quantiles can be computed by integrating the beta
  density
• For this example, prior and posterior
  distributions have same general form
• Priors which have the same form as the posteriors
  are called conjugate priors

16
Data example

• Maternal condition placenta previa
• Unusual condition of pregnancy where placenta is
  implanted very low in uterus preventing normal
  delivery
• Is this related to the sex of the baby?
• Proportion of female births in general population
  is 0.485
• Early study in Germany found that in 980 placenta
  previa births, 437 were female (0.4459)
• Ho p 0.485 versus Ha p lt 0.485

17
Placenta previa births

• Assume uniform prior Beta(1,1)
• Posterior is Beta(438,544)
• Posterior summaries
• Mean 0.446, Standard Deviation 0.016
• 95 confidence interval qbeta(.025,438,544),
• qbeta(.975,438,544) .415, .477

18
Sensitivity of Prior

• Suppose we took a prior more concentrated about
  the null
• hypothesis value
• E.g., Prior Normal(.485,.01)
• Posterior proportional to
• Constant of integration is about 10-294
• Mean, summary statistics, confidence intervals,
  etc., require numerical methods
• See S-script http//www.people.carleton.edu/rdob
  row/courses/275w05/Scripts/Bayes.ssc