Random Number Generation

1
Random Number Generation
2
Random Numbers in Simulation

• Necessary basic ingredient in simulation of each
  discrete systems
• Most computer languages have their own random
  number generator
• Two main properties
• Independence
• Uniformity
• How to make sure that random numbers are truly
  generated?

3
Dependence

• What are some examples of dependent random
  variables?
• Output of an M/M/1 queuing system
• Draws from a bowl of N colored chips.

4

• Each random number Ri is an independent sample
  drawn from a continuous uniform distribution
  between 0 and 1
• RiU(0,1)

f (x)
1
x
1
5
Properties

• Uniformity
• If the interval (0,1) is divided into n classes,
  or subintervals of equal length, the expected
  number of observations in each interval is N/n,
  where N is the total number of observations.
• Independence
• Probability of observing a value in a particular
  interval is independent of the previous values
  drawn.

6
Pseudo-Random Number Generation

• Why pseudo?
• The act of generating random numbers using a
  known method removes the potential for true
  randomness
• Method known random numbers replicated
• Goal produce a sequence of random numbers
  between 0 and 1 that imitates the ideal
  properties of U(0,1)

7
Deviations from True Randomness

• Generated numbers may not be uniformly
  distributed
• Generated numbers may be discrete-valued instead
  of continuous valued
• Mean of generated numbers may be too high or too
  low
• Variance of generated numbers may be too high or
  too low
• Dependence problems
• Auto-corrolation,
• Numbers successively higher than adjacent
  numbers,
• Several numbers above the mean followed by
  several numbers below the mean)

8
Deviations from True Randomness
9
Important Considerations

• Routine should be fast
• Select a computationally efficient method
• Routine should be portable different computers
• Produce same results wherever executed
• Routine should have a sufficiently long cycle
• Random numbers should be replicable
• Necessary for debugging and system comparisons
• Random numbers should closely approximate the
  properties of uniformity and independence

10
Techniques for RN Generation

• Linear Congruential Generator (LCG)
• Combined Multiple Recursive Generator (CMRG)
• More sophisticated techniques also exist

11
Linear Congruential Methods

• Produces a sequence of integers X1, X2, ..,
  between 0 and m-1 according to the following
  relationship
• X0 seed
• a constant multiplier
• c increment
• m modulus

12
Linear Congruential Methods

• When the increment c0, it is called
  multiplicative congruential method.
• When the increment c?0, it is called mixed
  congruential method.
• The choice of a, c, m, and X0 drastically affects
  the statistical properties and cycle length
• Exercise generate random numbers using linear
  congruential method for X027, a17, c43, and
  m100

13
Linear Congruential Methods

• Solution
• X0 27
• X1 (17.2743) mod 100 2
• R1 2/100 0.02
• X2 (17.243) mod 100 77
• R2 77/100 0.77
• X3 (17.7743) mod 100 52
• R3 52/100 0.52 ..

14
Linear Congruential Methods

• How closely generated random numbers approximate
  uniformity and independence?
• What are the maximum density and maximum period ?
• How large are the gaps on the 0,1 interval?
  (should not leave large gaps)
• How long are the cycle periods? (should avoid the
  recurrence of same sequence of generated numbers)

15
Linear Congruential Methods

• Using multiplicative congruential method, find
  period of the generator a13, m2664, and
  X01,2,3, and 4

16
Linear Congruential Methods

• Real LC random generators, take m to be at least
  231-12,147,483,647 (about 2.1 billion)
• The other parameters are carefully chosen to
  achieve full or nearly full cycle length.
• With a normal PC we can exhaust all these random
  numbers in a matter of minutes.
• So, for large application the cycle length might
  not sufficient.

17
Combined Multiple Recursive Generators (CMRG)

• For the simulation of real systems we usually
  need much longer periods
• A fruitful approach is to combine two or more
  multiplicative congruential generators to have
  good statistical properties and longer periods

18
Combined Multiple Recursive Generators (CMRG)
19
Combined Multiple Recursive Generators (CMRG)

• The parameters of the CMRG is carefully chosen to
  generate the proper statistical properties of
  random number
• Uniformity
• Independence
• The cycle length is huge 3.11057

20
Combined Multiple Recursive Generators (CMRG)

• The cycle length is huge 3.11057
• A normal PC can exhaust all these random numbers
  in 2.781040 millennia.
• Moores law The speed of computers double each
  one and a half year.
• It takes 216 years before a typical PC can
  exhaust all CMRG random numbers in one year.

21
Non-overlapping streams of RNs

• It is useful to separate the cycle of a RN
  generator into adjacent non-overlapping streams
  of RNs.
• The Arena generator has facility for splitting
  its cycle into 1. 81019 separate streams each
  of length 7.61038.
• There are sub-streams within each stream

22
Random Variates
23
Introduction

• Knowing how to
• generate random numbers that is uniformly
  distributed between 0 and 1,
• we now need to
• generate random number with other distribution
  functions.
• We want to transform samples from a uniform
  distribution between 0 and 1 into draws from a
  desired distribution. We refer to such draws as
  variates from this distribution

24
Continuous Distributions

• Uniform
• Exponential
• Gamma
• Weibull
• Normal
• Lognormal
• Beta
• Triangular

25
Discrete Distributions

• Bernoulli
• Discrete Uniform
• Binomial
• Geometric
• Negative Binomial
• Possion

26
Required Reading

• Read Appendix D of the text book
• Probability Distributions
• There will be question on the Midterm Exam
  regarding the applications of these probability
  distributions

27
Exponential Distribution

• Exponential Distribution
• Inter-arrival times of customers to a system that
  occurs at a constant rate
• The only continuous distribution with memoryless
  property
• Mean equal to standard deviation

28
Discrete Random Variates

• We show the technique through an example
• Consider the random variable X with the following
  PMF

29
Discrete Random Variates
First technique Dividing the 0,1 interval
0.1
0.5
0.4
0
0.1
0.6
1
X-2
X 0
X 3
30
Discrete Random Variates
F(x )
1.0
U
0.6
0.1
x
x
-2
3
Set X3
31
Discrete Uniform Random Variables

• Suppose X takes on values 1,,n with equal
  probability. In order to generate X based on U,
  we let
• if
• Hence, Xj if
• or

32
Class Activity

• Generate A Random Permutation Suppose we are
  interested in generating a permutation of numbers
  1,2,,n, which has n! possible orderings with
  equal probability
• Each group cannot be more than two students (if
  necessary only one group can be three)
• 10 minutes

33
Solution

• Let Pj j be the initial permutation
• Set kn
• Let
• Interchange the value of PI and Pk
• Let kk-1, if kgt1 goto step 3

34
Geometric Random Variables

• Suppose X is a geometric r.v. with parameter p,
  i.e., P(Xi)pqi-1, q1-p
• We can generate X by setting Xj if
• I.e.

35
The Inverse Transform Algorithm

• Proposition Let U be a uniform (0,1) r.v. For
  any continuous distribution function F, the r.v.
  X defined by
• XF-1(U)
• has distribution F.
• Proof.
• Since F is a monotone increasing function,

36
Continuous Random Variates

• Inverse Transform Technique
• Most straight forward, but not most efficient
• Compute CDF of desired random variable X
• Set F(X)U on the range of X
• Solve the equation F(X)U
• Generate as needed uniform random numbers and
  compute the desired random variates XiF-1(Ui)

37
Exponential Random Variates

• Inverse Transform Technique (example)
• The exponential pdf and cdf

38
Exponential Random Variates

• Inverse Transform Technique (example)
• What is the graphical presentation of this
  technique?

39
Triangular Random Variates

• Inverse Transform Technique (example)
• The Triangular pdf and cdf

40
Class Activity

• Derive the random variate formula for the
  triangular distribution
• Each group cannot be more than two students (if
  necessary only one group can be three)
• 15 minutes