General Principle of the Monte Carlo Method Spring 2005

1
General Principle of the Monte Carlo
MethodSpring 2005

• By Yaohang Li, Ph.D.
• Department of Computer Science
• North Carolina AT State University
• yaohang_at_ncat.edu

2
Review

• Last Class
• Nature of Monte Carlo
• Experimental Mathematics and Theoretical
  Mathematics
• Problems that Monte Carlo can handle
• Probabilistic
• Direct Simulation
• Deterministic
• Sophisticated Monte Carlo
• Monte Carlo Solution
• Good Experiment
• Good Conclusion
• Direct Simulation
• Comet Lifetime Problem
• This Class
• Assignment 1 is postponed to Feb. 8
• Monte Carlo Integration
• Next Class
• Variance Reduction Method

3
Numerical Integration

• Methods for approximating definite integrals
• Rectangle Rule
• Trapezoidal Rule
• Divide the curve into N strips of thickness
  h(b-a)/N
• Sum the area of each trip
• Approximate to that of a trapezium
• Simpsons Rule
• Calculate quadratic instead

4
Monte Carlo Method

• General Principles
• Every Monte Carlo computation that leads to
  quantitative results may be regarded as
  estimating the value of a multiple integral
• Efficiency
• Definition
• Suppose there are two Monte Carlo methods
• Method 1 n1 units of computing time, ?12
• Method 2 n2 units of computing time, ?22
• Methods comparison

5
Monte Carlo Integration

• Consider a simple integral
• Definition of expectation of a function on random
  variable ?
• If ? is uniformly distributed, then

6
Crude Monte Carlo

• Crude Monte Carlo
• If ?1, , ?n are independent random numbers
• Uniformly distributed
• then fif(?i) are random variates with
  expectation ?
• is an unbiased estimator of ?
• The variance is
• The standard error is
• ?/n1/2

7
Hit-or-Miss Monte Carlo Revisit

• Suppose 0 ? f(x) ? 1 when 0 ? x ? 1
• Main idea
• draw a curve yf(x) in the unit square 0?x,y?1
• is the proportion of the are of the square
  beneath the curve
• or we can write

8
Analysis of Hit-or-Miss Monte Carlo

• ? can be estimated as the a double integral
• The estimator of hit-or-miss Monte Carlo

9
Hit-or-Miss Monte Carlo

• Hit-or-Miss Monte Carlo
• We take n points at random in the unit square,
  and count the proportion of them which lie below
  the curve yf(x)
• The points are either in or out of the area below
  the curve
• The probability that a point lies under the curve
  is ?
• The Hit-or-Miss Monte Carlo is a Bernoulli trial
• the estimator of Hit-or-Miss Monte Carlo is
  binomial distributed

10
Binomial Distribution Revisit

• Binomial Distribution
• Discrete probability distribution Pp(nN) of
  obtaining exactly n successes out of N Bernoulli
  trials
• Each Bernoulli trial is true with probability p
  and false with probability q1-p
• variance N(1-p)p

11
Comparison of Hit-or-Miss Monte Carlo and Crude
Monte Carlo

• Standard error of Hit-or-Miss Monte Carlo
• Standard error of Crude Monte Carlo
• Hit-or-Miss Monte Carlo is always worse than
  Crude Monte Carlo
• Why?

12
Why Hit-or-Miss Monte Carlo is worse?

• Fact
• The hit-or-miss to crude sampling is equivalent
  to replacing g(x, ?) by its expectation f(x)
• The y variable in g(x,y) is a random variable
• Leads uncertainty
• Leads extra uncertainty in the final results
• Can be replace by exact value

13
General Principle of Monte Carlo

• If, at any point of a Monte Carlo calculation, we
  can replace an estimate by an exact value, we
  shall replace an estimate by an exact value, we
  shall reduce the sampling error in the final
  result

14
Summary

• Numerical Integration
• Monte Carlo Integration
• Crude Monte Carlo
• Hit-or-Miss Monte Carlo
• Comparison
• General Principle of Monte Carlo

15
What I want you to do?

• Review Slides
• Read the UNIX handbook if you are not familiar
  with UNIX
• Review basic probability/statistics concepts
• Work on your Assignment 1