Estimating Theoretical Moments and Parameters

1
Chapter 10

• Estimating Theoretical Moments and Parameters

2
II. The World Probability Theory (Ch. 6 8)
I. Data Descriptive Statistics (Ch. 2 5)
III. Drawing Conclusions Statistical
Inference (Ch. 9 11)
3

• Our main goal in this chapter is to learn to
  estimate various features of an underlying
  probability distribution (the world) on the
  basis of a given data set (the evidence).
• Our data set is only a (usually relatively
  small!) sample of all the possible and actual
  individuals in the population under study.
• Partial samples of the total population are
  almost always what we have to work with.
• But partial samples can be misleading!
• In order to do good science, we want to quantify
  how likely it is that our sample is misleading
• And we will need to get more precise about what
  misleading means here, too!

4

• We will want to know
• The value of the mean for the examination grades
• Whether the midterm grade was equal on average to
  the final grade
• The probability of AIDS occurring in a random
  selection of individuals and how accurate that
  number is
• The mean effect of weight on gasoline mileage
• The variance of interest rates about their mean
  value, a result that is important for evaluating
  portfolio risk

5

• We will want to know
• How often Old Faithful erupts and how much
  variation about that mean there is
• Whether the mean value of U.S. arms shipments
  exceeds those of Russia
• The mean horsepower of U.S. and foreign cars and
  how much variation there is about these means.

6

• All of these practical issues are examples of one
  general problem How can we obtain information
  about the parameters of distributions from
  observing data, and how can we assess the
  accuracy of that information?
• James Ramsey, p. 338

7

• It is important to distinguish

8

• For the rest of this course, whenever we have a
  collection of random variables
• We will assume that they are i.i.d.
• I.e, they are independently distributed, and they
  all come from the same probability distribution.

9

• The mean of the random variable is just the
  mean of the underlying parent distribution.

10

• Now lets examine the variance of
• Since the variables X1,,Xn are i.i.d., we can
  make use of what we saw in Chapter 7, namely

11
(No Transcript)
12

• In short, the variance of is 1/nth the
  variance of X.
• So, the standard deviation of is the size
  of the standard deviation of X.
• To sum up

13

• In other words, the standard deviation of the
  average of our sample (our data set) is only
  the size of the standard deviation of a sample
  of just one.
• So if you want your estimate to have 10 times
  less variability (measured by ?) in it than is
  present in American adults in general, you should
  take the average of 100 adults, instead of just
  measuring one adult.
• Thus, the reliability of our estimate increases
  dramatically as n increases.

14

• Notice that in our calculation of
• it was crucial that the Xis were all i.i.d.
  (independent and identically distributed)
• If the Xis were not independent, then we couldnt
  have made the step
• If the Xis were not identically distributed, then
  we couldnt have made the step

15

• Notice also that by the Central Limit Theorem,
  as n gets large, will approximate a Gaussian
  distribution.
• Knowing that has an (approximately) normal
  distribution gives us much useful information
  about how close to the mean of (and
  hence to the mean of X) we are likely to get with
  a given random sample.

16

• An estimator is consistent when
• for any ? gt0
• It is easy to see that, e.g., is consistent.
  
• The Law of Large Numbers says that

17

• The bias of an estimator is the difference
  between its expected value and the true value of
  the parameter it estimates
• When this difference is zero, the estimator is
  unbiased.
• E.g., is unbiased.

18

• What about the estimator of the variance
• ?
• It is easy to show that
• And that

19

• Thus, the term is an
  unbiased estimator of the variance, ?2, of a
  random variable.
• We denote this estimator by S2
• With s2 for

20

• This, by the way, is why Eviews likes to
  calculate the standard deviation as
• instead of
• Moreover, Eviews uses the same terminology

21

• The mean squared error (MSE) of an estimator is
• It is easy to see that the MSE of is just
  the variance of plus the square of its bias.
  

22
Confidence Intervals
23

• and S2 are unbiased estimators of the mean and
  variance of a distribution.
• They are point estimates
• They each supply a single unbiased estimation of
  the mean and variance of the parent distribution
• But they supply no information about how far
  off they are likely to be
• Because of the variability inherent in sampling,
  any given estimate is unlikely to be exactly
  right.

24

• It is often useful to supplement a particular
  numerical estimate with a confidence interval,
  which are useful because
• They supply information about how accurate a
  point estimate is likely to be.

25

• The population mean, ?, is some particular
  number, like 600, or 721.43.
• But we dont know what ? is, so we will have to
  estimate it from our data
• But our data could be a misleading
• sample means of various sizes are almost always
  different from the actual population mean.

26

• In short, there is always a certain amount of
  uncertainty about how close to ? our estimate
  is.
• Confidence Intervals have the form
• There is a 95 percent chance that will be
  less than ? and will be greater than ?
• ? is a fixed number, so and will contain
  some random variables.

27

• Stage 1 Calculating
• Stage 2 Determining a probability interval for Z
• Stage 3 Confidence intervals for ? when the
  variance ?2 is known
• Stage 4 Confidence intervals when ?2 is unknown
• Stage 5 Confidence intervals for proportions
• Stage 6 Confidence intervals for ?2

28
Stage 1 Calculating

• when Z N(0, 1)

29

• Lets ignore our original problem for the moment,
  and think instead about Z, where Z N(0, 1).
• Z is unlike X (and ), in that we know both
  the mean and variance of Z.
• In fact, we know everything there is to know
  about Z.
• Its distribution is

30

• Since we know everything about Z, we are able to
  calculate the value of

31

• Lets remind ourselves of this basic idea of
  using the cdf to calculate the probability of a
  random variable yielding a value in some
  particular interval -q, q.
• As an example, we will let q 1.96

32

• We start out with the N(0, 1) distribution

33

• We then calculate ?(1.96)

34

• We then calculate ?(1.96) .975

35

• Next we calculate ?(-1.96)

36

• Next we calculate ?(-1.96) .025

37

• Then we subtract
• ?(1.96) ?(-1.96)

38

• Then we subtract
• ?(1.96) ?(-1.96) .975 .025 .95

39

• To do this on Eviews, the crucial formula (for q
  1.96) is (_at_cnorm(1.96) - _at_cnorm(-1.96))
• This formula calculates ?(1.96) ?(-1.96)
• which we just saw is equal to
• What happens if we pick numbers other than 1.96?

40
Stage 2 Determining a probability interval for Z
41

• In Stage 1, we selected an interval -q, q, and
  calculated the probability that Z would lie in
  this interval.
• In our example where q 1.96, we took the
  termand solved for c in the equation
  (notice that c is determined by our choice of
  q 1.96).

42

• We now do the reverse. We select a value c (e.g.,
  c .95 or .99), and use that to determine the
  values of q and -q.
• For example, we will take the value c
  .95and calculate the values for q and -q
  which will determined by .95.
• cf. pp. 351 354 for some interesting discussion
  of the use of q and -q

43

• Thus, given our choice that c.95, we can
  calculate the value of q such that
• Similar to stage 1, but reversing the logic, this
  amounts to solving for q in

44

• When we pick a probability, and solve for the
  boundaries q, we are using the quantile function,
  and q is called a quantile.

45

• The quantile function for N(0,1) in Eviews is
• _at_qnorm(p) , where p ½(1 c)
• E.g. _at_qnorm(.025) , where c .95
• In words _at_qnorm(p) calculates the number n at
  which p of the area of the graph of the pdf of
  N(0, 1) lies to the left of n.
• In symbols _at_qnorm(p) calculates the n such that
  

46

• In pictures _at_qnorm(p) calculates the number n
  such that

The value of the function _at_qnorm(.025) is given
by this number 1.96
The area of the purple region is .025
47

• The graph of _at_qnorm(p) looks like
• Domain (0, 1) Range (?, ?)

48

• Compare the graphs of the quantile and the cdf of
  N(0, 1)

49

• So if we calculate _at_qnorm(.025), we get the
  number q such that
• One half of the remaining probability (.05)
  lies to one side of q, and the other half lies to
  the other side of q.
• So since c .95 is very close to 1, it will be
  very likely that Z will yield a value between q
  and q.

50

• In order to calculate our values for q, were
  going to make use of the fact that the normal
  distribution is symmetric about its mean.
• That is, for any value of q (q gt 0), there is the
  same amount of area in the normal distribution to
  the right of q as there is to the left of q

51

• We want to find a value of q such that the
  probability that Z yields a value between q and
  q is .95
• In other words, we want to find the q such that
  the area of the curve outside of the interval
  from q to q is .05

52

• Since the Normal distribution is symmetric
• To find our -q, we need to find that point on the
  x-axis where only .025 ½(.05) of the area of
  the curve is to the left of it.
• Similarly, q will be that point on the x-axis
  where only .025 of the area of the curve is to
  the right of it
• And thus, .975 ( 1 ½(.05) ) of the area is
  to the left of it

53

• Stage 3 Determining a confidence interval for
• When the variance ?2 is known

54

• Stages 1 and 2 showed us how to determine the
  size of the (symmetric) interval for Z

55

• But what about our target, a confidence interval
  for the unknown mean ? of X?
• We want to find something of the form

56

• We have already shown
• In Stage 2, we saw how to calculate q for a given
  choice of c (e.g., c .95)
• In Chapter 8, we saw that the Central Limit
  Theorem gives us

57

• So if n is sufficiently large, will
  be distributed pretty much like Z is, and so
  our conclusions about Z hold for it as well.
• So using what weve just done, we can pick a
  value c and use _at_qnorm to find the number q such
  that
• (In our example, c .95, and q 1.96)

58
(No Transcript)
59
(No Transcript)
60

• Thus, before we collect the actual values of our
  n data points, we know that with probability c,
  the mean ? that we are trying to find will lie in
  the interval
• This is the confidence interval at the chosen
  confidence level c (here c .95).
• The quantity is the margin of error.

61

• Here is how I want you to calculate a confidence
  interval at the confidence level c.
• Calculate or determine q, n, , and ?.
• Calculate
• Calculate and
• Conclude The confidence interval for ? at the c
  level of confidence is a, b, where a and b
  are from step 3.

62

• A confidence interval tells us the following
• For the fixed number ?
• The procedure has a 95 chance of producing two
  numbers a and b such that a ? b
• Once we get these numbers, were either right or
  wrong that a ? b
• And whether or not we are right is not something
  that we can know.
• We can only know the probability (e.g., 95)
• The length of the interval, (b a), gives us a
  measure of how uncertain any point estimate is.

63
Although our method captures the mean 95 of the
time, sometimes it will fail.

• 20 confidence intervals (at the .95) level

64

• To calculate the endpoints of a confidence
  interval in Eviews, you can simply use these
  formulas (for a random variable X, and a choice
  of c .95
• The lower bound is given by
• (_at_mean(x) - _at_qnorm(.975)_at_stdev(x)(_at_obs(x)(-.5))
  )
• The upper bound is given by
• (_at_mean(x) _at_qnorm(.975)_at_stdev(x)(_at_obs(x)(-.5))
  )

65

• Notice that when the variance is known, we often
  only need the N(0, 1) distribution when
  constructing confidence intervals for
  because, by the Central Limit Theorem
• If we use enough observations, should be
  approximately normally distributed.
• If is normally distributed, then its
  standardization has the N(0, 1)
  distribution.

66

• In real life, we rarely know the actual
  (population parameter) value ?.
• Instead, we have to estimate it from our data
  set.

67

• We developed our theory of confidence intervals
  using the assumption that we actually know the
  value of the standard deviation of the population
  of interest (given by X).
• Notice that this assumption plays a crucial role
  in our fundamental equation
• If you dont know what ? is, you cant produce
  this confidence interval!

68

• Fortunately, not all is lost.
• We have already seen that S2 is an unbiased
  estimator of ?2
• Were going to replace the unknown ? with S,
  which we can calculate, and get

69

• But notice that ? is a constant. It is some
  particular number, like 2, or 4.14573.
• S, however, is a random variable.
• It is built out of random variables.
• So before we run our experiment, S is not any
  particular number, it is just a variable with a
  distribution.
• So if we use S instead of ?, we have a second
  source of uncertainty in our equation.
• Now we have variation in possible values of both
• and S

70

• When the value of ? was assumed to be known, a
  crucial step in our generation of the confidence
  interval involved working with the
  standardization of
• Importantly, it is approximately true that

71

• But if we use S instead of ?, we will then be
  working with a different statistic
• Now there is some additional uncertainty in the
  denominator, because weve replaced the number ?
  with the random variable S.

72

• What are the consequences of admitting we only
  have S, not ??
• The variation in possible values of S is some
  extra uncertainty that we have so far ignored.
• Our distribution is less approximately Normal.
• In fact,
• is not distributed as approximately N(0, 1).
  Rather it has the Students T distribution.

73

• Conceptually, our procedure for determining a
  confidence interval (at some chosen confidence
  level c) is the same as before.
• However, this time, we must refer to the
  distribution of Students T, instead of that of
  N(0, 1).
• In order to get the right T-distribution, we will
  also need to provide one further piece of
  information
• the number of degrees of freedom of the sample,
  which is just n 1
• We note the distribution at T(n-1)
• For a sample of size 15, we would use T(14).

74

• As n (and hence n 1) gets very large, the T
  distribution gets arbitrarily close to being
  distributed as N(0, 1).
• But when n is fairly small, T can be importantly
  different from the normal distribution.
• Because the T-distribution accommodates the
  additional randomness contributed by the
  estimator S, it is a bit more spread out than
  N(0, 1), and so the confidence intervals will be
  larger.

75

• The T(5)
• compare with N(0, 1) in black

76

• The T distribution with 10 degrees of freedom
• compare with N(0, 1) in black

77

• The T distribution with 15 degrees of freedom
• compare with N(0, 1) in black

78

• The T distribution with 25 degrees of freedom
• compare with N(0, 1) in black

79

• Our two computing formulas are structurally the
  same as before
• But this time, q is the quantile of the
  T-distribution
• q _at_qtdist(.5(1 c), (n1)) q
  _at_qtdist(c .5(1 c), (n1))
• E.g. q _at_qtdist(.025, 19) q
  _at_qtdist(.975, 19)
• Some easy computing formulas are
• Lower bound
• (_at_mean(x) - _at_qtdist(.975,(_at_obs(x)-
  1))_at_stdev(x)(_at_obs(x)(-.5)))
• Upper bound
• (_at_mean(x) _at_qtdist(.975,(_at_obs(x)-1))_at_stdev(x)(_at_
  obs(x)(-.5)))

80

• What proportion of Americans approve of the
  Presidents performance?
• What proportion of customers used a coupon
  advertised on Amazon.com?

81

• Three facts make testing for proportions easy
• We can code all the responses as 0 or 1
• E.g., 1 if the customer used the coupon, 0
  otherwise 1 if the company went bankrupt, 0
  otherwise.
• If pr(X 1) ?, then ?2 ?(1 ?)
• So we can estimate the standard deviation for
  the proportion with
• p is the proportion of successes in our sample.

82

• Three facts make testing for proportions easy
• By the Central Limit Theorem, if n is fairly
  large, our estimate of p is nearly Normally
  distributed about p.
• So we can use the quantile function _at_qnorm to
  compute our confidence intervals
• For example

83

• Some easy computing formulas are
• (_at_mean(x) - (_at_qnorm(.975)_at_sqrt(((_at_mean(x)(1-_at_mea
  n(x)))/_at_obs(x)))))
• (_at_mean(x) (_at_qnorm(.975)_at_sqrt(((_at_mean(x)(1-_at_mea
  n(x)))/_at_obs(x)))))

84

• Conceptually, the construction of confidence
  intervals for our estimations of the variance is
  quite similar to those for the mean.

85

• If X1,,Xn are i.i.d. for some normal
  distribution, then has the
  chi-square distribution with n 1 degrees of
  freedom.
• Thus, we can use this distribution to calculate
  the probability

86

• The chi-square distribution (in blue) with 20
  degrees of freedom
• compare with a normal distribution in red

87

• But notice that this event can be easily
  transformed into something more useful
• Thus, we can use the quantile function for the
  chi-square distribution with n 1 degrees of
  freedom to determine a and b.
• Since this distribution is asymmetric, we will
  have to determine a and b separately.

88

• b is the upper quantile for a .95 confidence
  level, it is determined by
• _at_qchisq(.975, (_at_obs(x)-1))
• a is the lower quantile for a .95 confidence
  level, it is given by
• _at_qchisq(.025, (_at_obs(x)-1))
• The sum of squares is given by
• _at_sumsq(x)

89

• Thus, we can build up our confidence interval for
  Eviews
• The lower bound is given by
• (_at_sumsq(x)/_at_qchisq(.975, (_at_obs(x)-1)))
• The upper bound is given by
• (_at_sumsq(x)/_at_qchisq(.025, (_at_obs(x)-1)))

90

• Similarly, the square root of these bounds gives
  the confidence interval for the standard
  deviation
• The lower bound is given by
• _at_sqrt((_at_sumsq(x)/_at_qchisq(.975, (_at_obs(x)-1))))
• The upper bound is given by
• _at_sqrt((_at_sumsq(x)/_at_qchisq(.025, (_at_obs(x)-1))))