Statistics

1
Statistics

• Sampling Distributions and Point Estimation of
  Parameters

Contents, figures, and exercises come from the
textbook Applied Statistics and Probability for
Engineers, 5th Edition, by Douglas C. Montgomery,
John Wiley Sons, Inc., 2011.
2
Point Estimation

• Statistic
• A function of observations, , ,,
• Also a random variable
• Sample mean
• Sample variance
• Its probability distribution is called a sampling
  distribution

Contents, figures, and exercises come from the
textbook Applied Statistics and Probability for
Engineers, 5th Edition, by Douglas C. Montgomery,
John Wiley Sons, Inc., 2011.
3

• Point estimator of
• A statistic
• Point estimate of
• A particular numerical value

Contents, figures, and exercises come from the
textbook Applied Statistics and Probability for
Engineers, 5th Edition, by Douglas C. Montgomery,
John Wiley Sons, Inc., 2011.
4

• Mean
• The estimate
• Variance
• The estimate
• Proportion
• The estimate
• is the number of items that belong to the
  class of interest
• Difference in means,
• The estimate
• Difference in two proportions
• The estimate

5
Sampling Distributions and the Central Limit
Theorem

• Random sample
• The random variables , ,, are a
  random sample of size if (a) the s are
  independent random variables, and (b) every
  has the same probability distribution

6

• If , ,, are normally and independently
  distributed with mean and variance
• has a normal
  distribution
• with mean
• variance

Contents, figures, and exercises come from the
textbook Applied Statistics and Probability for
Engineers, 5th Edition, by Douglas C. Montgomery,
John Wiley Sons, Inc., 2011.
7

• Central Limit Theorem
• If , ,, is a random sample of size
  taken from a population (either finite or
  infinite) with mean and finite variance
  , and if is the sample mean, the limiting
  form of the distribution of
• as , is the standard normal
  distribution.
• Works when
• , regardless of the shape of the
  population
• , if not severely nonnormal

8

• Two independent populations with means and
  , and variances and
• is approximately standard normal, or
• is exactly standard normal if the two populations
  are normal

Contents, figures, and exercises come from the
textbook Applied Statistics and Probability for
Engineers, 5th Edition, by Douglas C. Montgomery,
John Wiley Sons, Inc., 2011.
9

• Example 7-1 Resistors
• An electronics company manufactures resistors
  that have a mean resistance of 100 ohms and a
  standard deviation of 10 ohms. The distribution
  of resistance is normal. Find the probability
  that a random sample of resistors will
  have an average resistance less than 95 ohms
• Example 7-2 Central Limit Theorem
• Suppose that has a continuous uniform
  distribution
• Find the distribution of the sample mean of a
  random sample of size

10

• Example 7-3 Aircraft Engine Life
• The effective life of a component used in a
  jet-turbine aircraft engine is a random variable
  with mean 5000 hours and standard deviation 40
  hours. The distribution of effective life is
  fairly close to a normal distribution. The engine
  manufacturer introduces an improvement into the
  manufacturing process for this component that
  increases the mean life to 5050 hours and
  decreases the standard deviation to 30 hours.
  Suppose that a random sample of
  components is selected from the old process and
  a random sample of components is
  selected from the improved process. What is the
  probability that the difference in the two sample
  means is at least 25 hours? Assume
  that the old and improved processes can be
  regarded as independent populations.

11

• Exercise 7-10
• Suppose that the random variable has the
  continuous uniform distribution
• Suppose that a random sample of
  observations is selected from this distribution.
  What is the approximate probability distribution
  of ? Find the mean and variance of this
  quantity.

Contents, figures, and exercises come from the
textbook Applied Statistics and Probability for
Engineers, 5th Edition, by Douglas C. Montgomery,
John Wiley Sons, Inc., 2011.
12
General Concepts of Point Estimation

• Bias of the estimator
• is an unbiased estimator if
• Minimum variance unbiased estimator (MVUE)
• For all unbiased estimator of , the one with
  the smallest variance
• is the MVUE for
• If , ,, are from a normal
  distribution with mean and variance

13

• Standard error of an estimator
• Estimated standard error
• or or
• If is normal with mean and variance
• and

Contents, figures, and exercises come from the
textbook Applied Statistics and Probability for
Engineers, 5th Edition, by Douglas C. Montgomery,
John Wiley Sons, Inc., 2011.
14

• Mean squared error of an estimate
• Relative efficiency of to

Contents, figures, and exercises come from the
textbook Applied Statistics and Probability for
Engineers, 5th Edition, by Douglas C. Montgomery,
John Wiley Sons, Inc., 2011.
15

• Example 7-4 Sample Mean and Variance Are Unbiased
• Suppose that is a random variable with mean
  and variance . Let , ,.,
  be a random sample of size from the population
  represented by . Show that the sample mean
  and sample variance are unbiased estimators
  of and , respectively.
• Example 7-5 Thermal Conductivity
• Ten measurements of thermal conductivity were
  obtained
• 41.60, 41.48, 42.34, 41.95, 41.86
• 42.18, 41.72, 42.26, 41.81, 42.04
• Show that and

16

• Exercise 7-31
• and are the sample mean and sample
  variance from a population with mean and
  variance . Similarly, and are the
  sample mean and sample variance from a second
  independent population with mean and variance
  . The sample sizes are and ,
  respectively.
• (a) Show that is an unbiased
  estimator of
• (b) Find the standard error of .
  How could you estimate the standard error?
• (c) Suppose that both populations have the same
  variance that is, . Show
  that
• Is an unbiased estimator of .

17
Methods of Point Estimation

• Moments
• Let , ,, be a random sample from
  the probability distribution , where
  can be a discrete probability mass function or a
  continuous probability density function. The th
  population moment (or distribution moment) is
  , 1, 2,. The corresponding th
  sample moment is 1, 2, .

Contents, figures, and exercises come from the
textbook Applied Statistics and Probability for
Engineers, 5th Edition, by Douglas C. Montgomery,
John Wiley Sons, Inc., 2011.
18

• Moment estimators
• Let , ,, be a random sample from
  either a probability mass function or a
  probability density function with unknown
  parameters , ,, . The moment
  estimators , ,, are found by
  equating the first population moments to the
  first sample moments and solving the
  resulting equations for the unknown parameters.

Contents, figures, and exercises come from the
textbook Applied Statistics and Probability for
Engineers, 5th Edition, by Douglas C. Montgomery,
John Wiley Sons, Inc., 2011.
19

• Maximum likelihood estimator
• Suppose that is a random variable with
  probability distribution , where is
  a single unknown parameter. Let , ,,
  be the observed values in a random sample of size
  . Then the likelihood function of the sample
  is
• Note that the likelihood function is now a
  function of only the unknown parameter . The
  maximum likelihood estimator (MLE) of is the
  value of that maximizes the likelihood
  function .

20

• Properties of a Maximum Likelihood Estimator
• Under very general and not restrictive
  conditions, when the sample size is large and
  if is the maximum likelihood estimator of
  the parameter ,
• (1) is an approximately unbiased estimator for
  
• (2) the variance of is neatly as small as
  the variance that could be obtained with any
  other estimator, and
• (3) has an approximate normal distribution.

Contents, figures, and exercises come from the
textbook Applied Statistics and Probability for
Engineers, 5th Edition, by Douglas C. Montgomery,
John Wiley Sons, Inc., 2011.
21

• Invariance property
• Let , ,., be the maximum
  likelihood estimators of the parameters ,
  , , . Then the maximum likelihood
  estimator of any function
  of these parameters is the same function
• of the estimators , ,, .

Contents, figures, and exercises come from the
textbook Applied Statistics and Probability for
Engineers, 5th Edition, by Douglas C. Montgomery,
John Wiley Sons, Inc., 2011.
22

• Bayesian estimation of parameters
• Sample , ,,
• Joint probability distribution
• Prior distribution for
• Posterior distribution for
• Marginal distribution

23

• Example 7-6 Exponential Distribution Moment
  Estimator
• Suppose that , ,, is a random
  sample from an exponential distribution with
  parameter . For the exponential,
  .
• Then results in
  .
• Example 7-7 Normal Distribution Moment Estimators
• Suppose that , ,, is a random
  sample from a normal distribution with parameters
  and . For the normal distribution,
  and
• . Equating to
  and to gives
• and
• Solve these equations.

24

• Example 7-8 Gamma Distribution Moment Estimators
• Suppose that , ,, is a random
  sample from a gamma distribution with parameters
  and , For the gamma distribution,
  and
• . Solve
• and

Contents, figures, and exercises come from the
textbook Applied Statistics and Probability for
Engineers, 5th Edition, by Douglas C. Montgomery,
John Wiley Sons, Inc., 2011.
25

• Example 7-9 Bernoulli Distribution MLE
• Let be a Bernoulli random variable. The
  probability mass function is
• where is the parameter to be estimated. The
  likelihood function of a random sample of size
  is
• Find that maximizes .

Contents, figures, and exercises come from the
textbook Applied Statistics and Probability for
Engineers, 5th Edition, by Douglas C. Montgomery,
John Wiley Sons, Inc., 2011.
26

• Example 7-10 Normal Distribution MLE
• Let be normally distributed with unknown
  and known variance . The likelihood
  function of a random sample of size , say
  , ,, , is
• Find .
• Example 7-11 Exponential Distribution MLE
• Let be exponentially distributed with
  parameter . The likelihood function of a
  random sample of size , say , ,,
  , is
• Find .

27

• Example 7-12 Normal Distribution MLEs for
  and
• Let be normally distributed with mean
  and variance , where both and are
  unknown. The likelihood function of a random
  sample of size is
• Find and .
• Example 7-13
• From Example 7-12, to obtain the maximum
  likelihood estimator of the function
• Substitute the estimators and into the
  function , which yields

28

• Example 7-14 Uniform Distribution MLE
• Let be uniformly distributed on the interval
  0 to . Since the density function is
  for and zeros otherwise, the
  likelihood function of a random sample of size
  is
• for , ,,
• Find .

Contents, figures, and exercises come from the
textbook Applied Statistics and Probability for
Engineers, 5th Edition, by Douglas C. Montgomery,
John Wiley Sons, Inc., 2011.
29

• Example 7-15 Gamma Distribution MLE
• Let , ,, be a random sample from
  the gamma distribution. The log of likelihood
  function is
• Find that

Contents, figures, and exercises come from the
textbook Applied Statistics and Probability for
Engineers, 5th Edition, by Douglas C. Montgomery,
John Wiley Sons, Inc., 2011.
30

• Example 7-16 Bayes Estimator for the Mean of a
  Normal Distribution
• Let , ,, be a random sample from
  the normal distribution with mean and
  variance , where is unknown and
  is known. Assume that the prior distribution for
  is normal with mean and variance
  that is,
• The joint probability distribution of the sample
  is

Contents, figures, and exercises come from the
textbook Applied Statistics and Probability for
Engineers, 5th Edition, by Douglas C. Montgomery,
John Wiley Sons, Inc., 2011.
31

• Show that
• Then the Bayes estimate of is

Contents, figures, and exercises come from the
textbook Applied Statistics and Probability for
Engineers, 5th Edition, by Douglas C. Montgomery,
John Wiley Sons, Inc., 2011.
32

• Exercise 7-42
• Let , ,, be uniformly distributed
  on the interval 0 to . Recall that the
  maximum likelihood estimator of is
  .
• (a) Argue intuitively why cannot be an
  unbiased estimator for .
• (b) Suppose that . Is
  it reasonable that consistently
  underestimates ? Show that the bias in the
  estimator approaches zero as gets large.
• (c) Propose an unbiased estimator for .
• (d) Let . Use the fact that
  if and only if each to
  derive the cumulative distribution function of
  . Then show that the probability density
  function of is

33

• Use this result to show that the maximum
  likelihood estimator for is biased.
• (e) We have two unbiased estimators for the
  moment estimator and
  
• , where
  is the largest observation in a random
  sample of size . It can be shown that
  and that
• . Show that if
  , is a better estimator than .
  In what sense is it a better estimator of ?

Contents, figures, and exercises come from the
textbook Applied Statistics and Probability for
Engineers, 5th Edition, by Douglas C. Montgomery,
John Wiley Sons, Inc., 2011.
34

• Exercise 7-50
• The time between failures of a machine has an
  exponential distribution with parameter .
  Suppose that the prior distribution for is
  exponential with mean 100 hours. Two machines
  are observed, and the average time between
  failures is hours.
• (a) Find the Bayes estimate for .
• (b) What proportion of the machine do you think
  will fail before 1000 hours?

Contents, figures, and exercises come from the
textbook Applied Statistics and Probability for
Engineers, 5th Edition, by Douglas C. Montgomery,
John Wiley Sons, Inc., 2011.
35
Contents, figures, and exercises come from the
textbook Applied Statistics and Probability for
Engineers, 5th Edition, by Douglas C. Montgomery,
John Wiley Sons, Inc., 2011.