Bayesian Data Analysis

1
Bayesian Data Analysis

• Eric T. Bradlow
• Associate Professor of Marketing and Statistics
• The Wharton School
• Lecture 2

2
Todays Lecture

• Question and Answers from Previous Lecture
• All you wanted to know about distributions (and
  more)
• A return to the normal-normal model
• The introduction of the Poisson-Gamma model
• Final Thoughts

3
Q A from previous lecture
Q and A session 1
4
All you wanted to know about distributions

• Prior p(?)
• Prior Knowledge/subjective beliefs about ?
• Likelihood p(Y?)
• Data Generation Process of Y given ?
• Posterior distribution p(?Y) p(?)p(Y
  ?)/p(Y)
• Updated Knowledge/subjective beliefs about ?
  after observing Y
• How can you do simulation from this distribution?
• Marginal distribution or the prior predictive
  distribution
• p(Y) ? p(Y ?) p(?) d?
• The probability of the data, under the model
• How can you compute this via simulation?

5
All you wanted to know about distributions(and
more)-continued

• Predictive distribution
• Predictions in the Bayesian setting are obtained
  from this distribution
• How would you do this via simulation?
• Marginal Posterior Distribution
• This is the main reason you need Bayesian
  computational procedures. You would like to
  integrate the joint posterior with respect to
  some set of variables however, such integration
  is not doable in closed-form.

6
A return to the normal-normal model
Likelihood
Prior
Posterior of ?
Marginal Distribution

• Also known as the histogram or data-fitting
  distribution
• How could you do estimation given this model????

Predictive Distribution
Notice the relationships among the variances of
these distributions
7
Comments about the Normal-Normal Model

• This is the most fundamental of all models, and
  when the variances are unknown even this model
  does not lend it self to closed-form solutions
• When inference is desired for the variances of
  the normal model, the classic framework is to
  assume that
• Which leads to the following two well-known
  results
• (a) The marginal distribution of Y, integrated
  over ?2 is a t-distribution (a gamma mixture of
  normals is t.
• (b) The posterior distribution of ?2 is also
  inverse gamma, where the parameters are nv, v?
  ns2/(vn) (observations add and SSE add).

8
The Poisson-Gamma Model

• A very common distribution for integer count data
  is the Poisson distribution.
• The Poisson distribution arises naturally when
• Arrivals in any given non-overlapping time slices
  are independent
• The timing of arrivals happens according to an
  exponential timing process
• Then the COUNT of the number of arrivals in a
  fixed time period is Poisson.

9
Properties of the Poisson Model

• When counts follow the Poisson model, we say
• X1,Xi, , XnPoisson(?i)
• E(Xi) ?i, Var(Xi) ?i
• Notice for the Poisson distribution, the mean and
  variance are the same. This is very restrictive,
  and leads to some very simple tests for the
  appropriateness of the Poisson model for count
  data.
• Another comment to assess the Poisson-ness of
  data is to look at P(Xx)/P(Xx1) for the
  Poisson which is (x1)/ ? which is linear in X.
  So people sometimes plot this ratio of successive
  bins against x to see if its linear. Note that
  the 1/slope of this plot is an estimate of ?.

10
Poisson Pictures
Normal Approximation to the Poisson is used
quite often.
11
The Hyper-dispersed Poisson

• As mentioned previously, the mean and variance of
  the Poisson being the same is restrictive.
• Classic solution is to assume that
• Then, to compute the marginal distribution of Xi
  we integrate the Poisson(?i) with respect to the
  Gamma mixing distribution, and we get that
• Note, that just like the normal-normal model
  leads to a hyper-dispersed normal. This leads to
  a hyper-dispersed Poisson, also known as the
  Negative-Binomial distribution.
• Also note that the Posterior for ?i takes a nice
  form.

12
Summary of Todays Lecture

• To really be able to model and estimate data
  appropriately, you need to understand what I
  call distributional arithmetic.
• Bayesian computation, which we will get to, is
  all based on distributional arithmetic, i.e. the
  ability to marginalize distributions, write down
  conditional distributions and posteriors.
• Next Class
• Basics of IRT Models
• Assignment Email to me
• (a) One question about Ch1, CH2, or lectures
• (b) Your most pressing question about IRT models