Point Estimation of Parameters

1
Chapter 7

• Point Estimation of Parameters

2
Learning Objectives

• Explain the general concepts of estimating
• Explain important properties of point estimators
• Know how to construct point estimators using the
  method of maximum likelihood
• Understand the central limit theorem
• Explain the important role of the normal
  distribution

3
Statistical Inference

• Used to make decisions or to draw conclusions
  about a population
• Utilize the information contained in a sample
  from the population
• Divided into two major areas
• parameter estimation
• hypothesis testing
• Use sample data to compute a number
• Called a point estimate

4
Statistic and Sampling Distribution

• Obtain a point estimate of a population parameter
• Observations are random variables
• Any function of the observation, or any
  statistic, is also a random variable
• Sample mean and sample variance are statistics
• Has a probability distribution
• Call the probability distribution of a statistic
  a sampling distribution

5
Definition of the Point Estimate

• Suppose we need to estimate the mean of a single
  population by a sample mean
• Population mean, ?, is the unknown parameter
• Estimator of the unknown parameter is the sample
  mean
• is a statistic and can take on any value
• Convenient to have a general symbol
• Symbols are used in parameter estimation
• Unknown population parameter id denoted by
• Point estimate of this parameter by
• Point estimator is a statistic and is denoted by

6
General Concepts of Point Estimation

• Unbiased Estimator
• Estimator should be close to the true value of
  the unknown parameter
• Estimator is unbiased when its expected value is
  equal to the parameter of interest
• Bias is zero
• Variance of a Point Estimator
• Considering all unbiased estimators, the one with
  the smallest variance is called the minimum
  variance unbiased estimator (MVUE)
• MVUE is most likely estimator that gives a close
  value to the true value of the parameter of
  interest

7
Standard Error

• Measure of precision can be indicated by the
  standard error
• Sampling from a normal distribution with mean ?
  and variance ?2
• Distribution of is normal with mean ? and
  variance ?2/n
• Standard error of
• Not know ?, we will substitute the s into the
  above equation

8
Mean Square Error (MSE)

• It is necessary to use a biased estimator
• Mean square error of the estimator can be used
• Mean square error of an estimator is difference
  between the estimator and the unknown parameter
  
• An estimator is an unbiased estimator
• If the MSE of the estimator is equal to the
  variance of the estimator
• Bias is equal to zero

Eq.7-3
9
Relative Efficiency

• Suppose we have two estimators of a parameter
  with their corresponding mean square errors
• Defined as
• If this relative efficiency is less than 1
• Conclude that the first estimator give us a more
  efficient estimator of the unknown parameter than
  the second estimator
• Smaller mean square error

10
Example

• Suppose we have a random sample of size 2n from a
  population denoted by X, and E(X) ? and V(X) ?2
• Let
• be two estimators of ?
• Which is the better estimator of ? Explain your
  choice.

11
Solution

• Expected values are
• and are unbiased estimators of ?
• Variances are
• MSE
• Conclude that is the better estimator with
  the smaller variance

12
Methods of Point Estimation

• Definition of unbiasness and other properties do
  not provide any guidance about how good
  estimators can be obtained
• Discuss the method of maximum likelihood
• Estimator will be the value of the parameter that
  maximizes the probability of occurrence of the
  sample values

13
Definition

• Let X be a random variable with probability
  distribution f(x?)
• ? is a single unknown parameter
• Let x1, x2, , xn be the observed values in a
  random sample of size n
• Then the likelihood function of the sample
• L(?) f(x1?). f(x2?). f(xn?)

14
Sampling Distribution of Mean

• Sample mean is a statistic
• Random variable that depends on the results
  obtained in each particular sample
• Employs a probability distribution
• Probability distribution of is a sampling
  distribution
• Called sampling distribution of the mean

15
Sampling Distributions of Means

• Determine the sampling distribution of the sample
  mean
• Random sample of size n is taken from a normal
  population with mean ? and variance ?2
• Each observation is a normally and independently
  distributed random variable with mean ? and
  variance ?2

16
Sampling Distributions of Means-Cont.

• By the reproductive property of the normal
  distribution
• X-bar has a normal distribution with mean
• Variance

17
Central Limit Theorem

• Sampling from an unknown probability distribution
• Sampling distribution of the sample mean will be
  approximately normal with mean ? and variance
  ?2/n
• Limiting form of the distribution of
• Most useful theorems in statistics, called the
  central limit theorem
• If n ? 30, the normal approximation will be
  satisfactory regardless of the shape of the
  population

18
Two Independent Populations

• Consider a case in which we have two independent
  populations
• First population with mean ?1 and variance ?21
  and the second population with mean ?2 and
  variance ?22
• Both populations are normally distributed
• Linear combinations of independent normal random
  variables follow a normal distribution
• Sampling distribution of is
  normal with mean
• and variance