010'141 Engineering Mathematics II Lecture 10 Limit Theorems

1
010.141 Engineering Mathematics IILecture
10Limit Theorems

• Bob McKay
• School of Computer Science and Engineering
• College of Engineering
• Seoul National University

2
Outline

• Markovs Inequality
• Chebyshevs Inequality
• Weak Law of Large Numbers
• The Central Limit Theorem
• Strong Law of Large Numbers
• Convex Functions and Jensens Inequality

3
Markovs Inequality

• If X is a non-negative random variable, then for
  any a gt 0,
• PX gt a ? EX / a
• Or to put it another way, the expectation divided
  by a underestimates the probability that X
  exceeds a

4
Chebyshevs Inequality

• If X is a random variable, with mean ? and
  variance ?2, then for any k gt 0
• PX - ? ? k ? ?2/k2
• That is, the probability that the variance from
  the mean exceeds k is bounded above by (?/k)2

5
The Weak Law of Large Numbers

• Let X1, X2, X3 be a sequence of independent
  random variables with identical distributions of
  mean ?. Then for any ? gt 0,
• P(X1 X2 Xn)/n - ? ? ? ? 0 as n ? ?
• In the limit for large numbers of samples, the
  sample mean tends to the distribution mean

6
The Central Limit Theorem

• Let X1, X2, X3 be a sequence of independent
  random variables with identical distributions of
  mean ? and variance ?2. Then for any -? lt a lt ?
• P(X1 X2 Xn - n?)/??n ?a ? 1/?2?
  ?-?a e-x2/2 dx as n ? ?
• In the limit, the distribution of standardised
  means tends to the normal

7
Central Limit Theorem Application

• The number of students who enrol in Engineering
  Maths II is a Poisson random variable with mean
  20
• In any year, the class will be split into two if
  more than 30 take the class
• What is the probability that the class will be
  split into two in a given year

8
Central Limit Theorem Application

• Its very difficult to calculate this tail of the
  Poisson distribution exactly
• However the sum of Poisson distributions is
  Poisson, so we can regard this distribution as
  the sum of 20 Poisson distributions, each with
  mean 1
• Then we can apply the Central Limit Theorem, and
  approximate this distribution with a normal
  distribution, and calculate instead the tail of
  the normal distribution.

9
The Strong Law of Large Numbers

• Let X1, X2, X3 be a sequence of independent
  random variables with identical distributions of
  mean ?. Then with probability 1,
• (X1 X2 Xn)/n ? ? as n ? ?
• In the limit for large numbers of samples, the
  sample mean will differ from the distribution
  mean by more than a particular given amount only
  a finite number of times

10
Convex and concave functions

• A function f(x) which is twice differentiable is
  convex if f(x) ? 0 for all x
• A function f(x) which is twice differentiable is
  concave if f(x) ? 0 for all x

11
Jensens Inequality

• If f(x) is a convex function, then
• E f(X) ? f(E X)
• on the assumption that these expectations exist
  and are finite

12
Summary

• Markovs Inequality
• Chebyshevs Inequality
• Weak Law of Large Numbers
• The Central Limit Theorem
• Strong Law of Large Numbers
• Convex Functions and Jensens Inequality

13
?????