STATISTICAL%20INFERENCE%20PART%20III

1
STATISTICAL INFERENCEPART III

• EXPONENTIAL FAMILY, LOCATION AND SCALE PARAMETERS

2
EXPONENTIAL CLASS OF PDFS

• X is a continuous (discrete) rv with pdf f(x?),
  ???. If the pdf can be written in the following
  form

then, the pdf is a member of exponential class of
pdfs of the continuous (discrete) type. (Here, k
is the number of parameters)
3
REGULAR CASE OF THE EXPONENTIAL CLASS OF PDFS

• We have a regular case of the exponential class
  of pdfs of the continuous type if
• Range of X does not depend on ?.
• c(?) 0, w1(?),, wk(?) are real valued
  functions of ? for ???.
• h(x) 0, t1(x),, tk(x) are real valued
  functions of x.

If the range of X depends on ?, then it is called
irregular exponential class or range-dependent
exponential class.
4
EXAMPLES

• XBin(n,p), where n is known. Is this pdf a
  member of exponential class of pdfs? Why?

Binomial family is a member of exponential family
of distributions.
5
EXAMPLES

• XCauchy(1,?). Is this pdf a member of
  exponential class of pdfs? Why?

Cauchy is not a member of exponential family.
6
REGULAR CASE OF THE EXPONENTIAL CLASS OF PDFS

• Random Sample from Regular Exponential Class

is a css for ?.
If Y is an UE of ?, Y is the MVUE of ?.
7
EXAMPLES

• Recall XBin(n,p), where n is known.

This family is a member of exponential family of
distributions.
is a CSS for p.
is UE of p.
is MVUE of p.
8
EXAMPLES

• XN(?,?2) where both ? and ?2 is unknown. Find a
  css for ? and ?2 .

9
FISHER INFORMATION AND INFORMATION CRITERIA

• X, f(x?), ???, x?A (not depend on ?).

Definitions and notations
10
FISHER INFORMATION AND INFORMATION CRITERIA

The Fisher Information in a random variable X
The Fisher Information in the random sample
Lets prove the equalities above.
11
FISHER INFORMATION AND INFORMATION CRITERIA

12
FISHER INFORMATION AND INFORMATION CRITERIA

13
FISHER INFORMATION AND INFORMATION CRITERIA

The Fisher Information in a random variable X
The Fisher Information in the random sample
Proof of the last equality is available on
Casella Berger (1990), pg. 310-311.
14
CRAMER-RAO LOWER BOUND (CRLB)

• Let X1,X2,,Xn be sample random variables.
• Range of X does not depend on ?.
• YU(X1,X2,,Xn) a statistic not containing ?.
• Let E(Y)m(?).
• Z?'(x1,x2,,xn?) is a r.v.
• E(Z)0 and V(Z)In(?) (from previous slides).
• Let prove this!

15
CRAMER-RAO LOWER BOUND (CRLB)

• Cov(Y,Z)E(YZ)-E(Y)E(Z)E(YZ)

16
CRAMER-RAO LOWER BOUND (CRLB)

• E(Y.Z)m(?)
• -1?Corr(Y,Z)?1?
• 0 ?Corr(Y,Z)2?1 ?

The Cramer-Rao Inequality (Information Inequality)
17
CRAMER-RAO LOWER BOUND (CRLB)

• CRLB is the lower bound for the variance of the
  unbiased estimator of m(?).
• When V(Y)CRLB, Y is the MVUE of m(?).
• For a r.s., remember that In(?)n I(?), so,

18
EFFICIENT ESTIMATOR

• T is an efficient estimator (EE) of ? if
• T is UE of ?, and,
• Var(T)CRLB
• Y is an efficient estimator (EE) of its
  expectation, m(?), if its variance reaches the
  CRLB.
• An EE of m(?) may not exist.
• The EE of m(?), if exists, is unique.
• The EE of m(?) is the unique MVUE of m(?).

19
ASYMPTOTIC EFFICIENT ESTIMATOR

• Y is an asymptotic EE of m(?) if

20
EXAMPLES

• A r.s. of size n from XPoi(?).
• Find CRLB for any UE of ?.
• Find UMVUE of ?.
• Find an EE for ?.
• Find CRLB for any UE of exp-2?. Assume n1, and
  show that is UMVUE of exp-2?. Is this a
  reasonable estimator?

21
EXAMPLE

• A r.s. of size n from XExp(?). Find UMVUE of ?,
  if exists.

22
LIMITING DISTRIBUTION OF MLEs

• MLE of ?
• X1,X2,,Xn is a random sample.

23
LIMITING DISTRIBUTION OF MLEs

• Let be MLEs of ?1, ?2,, ?m.

• If Y is an EE of ?, then ZabY is an EE of
  abm(?) where a and b are constants.

24
LOCATION PARAMETER

• Let f(x) be any pdf. The family of pdfs f(x??)
  indexed by parameter ? is called the location
  family with standard pdf f(x) and ? is the
  location parameter for the family.
• Equivalently, ? is a location parameter for f(x)
  iff the distribution of X?? does not depend on ?.

25
Example

• If XN(?,1), then X-?N(0,1) ? distribution is
  independent of ?. ? ? is a location parameter.
• If XN(0,?), then X-?N(-?,?) ? distribution is
  NOT independent of ?. ? ? is NOT a location
  parameter.

26
LOCATION PARAMETER

• Let X1,X2,,Xn be a r.s. of a distribution with
  pdf (or pmf) f(x ?) ???. An estimator
  t(x1,,xn) is defined to be a location
  equivariant iff
• t(x1c,,xnc) t(x1,,xn) c
• for all values of x1,,xn and a constant c.
• t(x1,,xn) is location invariant iff
• t(x1c,,xnc) t(x1,,xn)
• for all values of x1,,xn and a constant c.

Invariant does not change
27
Example

• Is location invariant or equivariant
  estimator?
• Let t(x1,,xn) . Then,
• t(x1c,,xnc) (x1cxnc)/n (x1xnnc)/n
  c t(x1,,xn) c
• ? location equivariant

28
Example

• Is s² location invariant or equivariant
  estimator?
• Let t(x1,,xn) s²
• Then,
• t(x1c,,xnc)
• t(x1,,xn)? Location invariant

(x1,,xn) and (x1c,,xnc) are located at
different points on real line, but spread among
the sample values is same for both samples.
29
SCALE PARAMETER

• Let f(x) be any pdf. The family of pdfs f(x/?)/?
  for ?gt0, indexed by parameter ?, is called the
  scale family with standard pdf f(x) and ? is the
  scale parameter for the family.
• Equivalently, ? is a scale parameter for f(x) iff
  the distribution of X/? does not depend on ?.

30
Example

• Let XExp(?). Let YX/?.
• You can show that f(y)exp(-y) for ygt0
• Distribution is free of ?
• ? is scale parameter.

31
SCALE PARAMETER

• Let X1,X2,,Xn be a r.s. of a distribution with
  pdf (or pmf) f(x ?) ???. An estimator
  t(x1,,xn) is defined to be a scale equivariant
  iff
• t(cx1,,cxn) ct(x1,,xn)
• for all values of x1,,xn and a constant cgt0.
• t(x1,,xn) is scale invariant iff
• t(cx1,,cxn) t(x1,,xn)
• for all values of x1,,xn and a constant cgt0.

32
Example

• Is scale invariant or equivariant estimator?
• Let t(x1,,xn) . Then,
• t(cx1,,cxn) c(x1xn)/n c ct(x1,,xn)
• ? Scale equivariant

33
LOATION-SCALE PARAMETER

• Let f(x) be any pdf. The family of pdfs
• f((x??) /?)/? for ?gt0, indexed by parameter
  (?,?), is called the location-scale family with
  standard pdf f(x) and ? is a location parameter
  and ? is the scale parameter for the family.
• Equivalently, ? is a location parameter and ? is
  a scale parameter for f(x) iff the distribution
  of (X??)/? does not depend on ? and?.

34
Example

• 1. XN(µ,s²). Then, Y(X- µ)/s N(0,1)
• Distribution is independent of µ and s²
• µ and s² are location-scale paramaters
• 2. XCauchy(?,ß). You can show that the p.d.f. of
  Y(X- ß)/ ? is f(y) 1/(p(1y²))
• ? ß and ? are location-and-scale parameters.

35
LOCATION-SCALE PARAMETER

• Let X1,X2,,Xn be a r.s. of a distribution with
  pdf (or pmf) f(x ?) ???. An estimator
  t(x1,,xn) is defined to be a location-scale
  equivariant iff
• t(cx1d,,cxnd) ct(x1,,xn)d
• for all values of x1,,xn and a constant cgt0.
• t(x1,,xn) is location-scale invariant iff
• t(cx1d,,cxnd) t(x1,,xn)
• for all values of x1,,xn and a constant cgt0.