Performance Engineering

1
Performance Engineering
Simulation

• Prof. Jerry Breecher

2
Simulation

• OVERVIEW
•  
• A simulation is any action that mimics reality.
• Chess simulates war.
• Monopoly simulates real estate investment.
• Any game is, in fact, a model of a reality.
•  
• Example
•  
• Computers simulate monopoly (that simulates ....)
• Computers simulate population growth/energy
  supplies.
• Computers simulate expansion of the Universe.
• Computers can simulate computers!!

3
Simulation
Why Do Simulation

•  PROS AND CONS OF SIMULATIONS
•  
•  
• The model can be as complicated as we wish (
  analytical models are limited by what we can
  write as an equation.)
• Must iterate a simulation many times - may take a
  long compute time.
• Simulations can be expensive to set up.
• Analytical modeling may be more compact and
  simple.

4
Simulation
Pitfalls

• COMMON MISTAKES
•  
• Since modeling is not something to dive into
  lightly, what warnings are needed up-front?
•  
• Engineers inherently dont trust models. We tend
  to believe only things we can see. So you need
  to generate belief in your model among your
  customers. Be very careful giving out answers
  to your model dont do so before you believe
  the results yourself. Here are a number of
  issues you need to take into account
•  
• You need to get the level of detail right! Its
  very hard to know how much detail you need.
  Theres a tendency among engineers for too much
  detail after all, were used to writing code.
  At the other extreme too little detail can
  leave valuable mechanisms unaccounted for.
• Unverified or invalid models. You need to be
  constantly checking that your model matches
  reality. This is not easy. It requires finding
  things you can measure such as the last version
  of the mechanism youre modeling.

5
Simulation
Pitfalls

• COMMON MISTAKES
•  
• The language of the simulation. If the innards
  of the model require a lot of code (random number
  generators, event handlers, etc.) you can waste a
  great deal of time writing your own. Conversely,
  simple models dont need fancy simulation
  packages. More on this later.
•  
• Mechanical Issues. The simulation didnt run
  long enough to be able to get the number of
  samples you need or to avoid end conditions.
  Your random number generator wasnt really
  random.
•  

6
Simulation
Pitfalls

• COMMON MISTAKES
•  
• Software Engineering Issues. In this management
  category falls things like
•  
• Not knowing how long the model will take.
• Lack of good software design. What starts out as
  a simple hacked together program becomes more
  and more complex and more and more unsupportable.
• Not clearly defining the goals so you know when
  youre done someone always will ask a what if
  question that will keep you going forever.
• Inadequate customer buy-in. Your customers
  dont or wont believe the outcome.
• Inexplicable results.

7
Simulation
Simulation Lingo

• This section describes briefly some of the terms
  used in Simulation.
•  
• State Variables All the program variables needed
  to define where the simulation is at time T.
•  
• Event A change in the system state. Arrivals
  and departures from any center is an event.
•  
• Time Driven Simulations Where time is what
  matters. The state variables are defined in
  terms of time. They can be either continuous or
  discrete.
•  
• Event Driven Simulations Where changes of state
  (events) are what matters. State variables are
  defined in terms of these events.

8
Simulation
Simulation Lingo

• This section describes briefly some of the terms
  used in Simulation.
•  
• Deterministic or Probabilistic Every time you
  run the model, do you get the same answer or a
  different answer. Either can be correct, but you
  should know beforehand what to expect. With a
  probabilistic model, running it longer doesnt
  give it a better result.
•  
• Open and Closed Open means jobs/inputs are
  coming in external to the model. Closed means
  all jobs recycle through the model.
•  
• Stable and Unstable Does the model converge to
  some answer or not.
•  

9
Simulation
Simulation Languages

•  Heres how these various languages/techniques
  compare

Special Simulation Language Event Management, Random Variables Screen Based Animation Language Based Subroutines
SimScript II.5 English Like Yes Yes Yes
GPSS Yes Yes Yes
SSim No Yes ? Yes
Homebrew C
10
Simulation
Types of Simulations

• TRACE-DRIVEN SIMULATIONS
•   
• You gather, independently, from some real
  environment, a set of inputs. Then you play it
  back through the simulator.
• Nice in that you can keep your inputs constant
  and change the way the system handles those
  inputs.
• You can verify that your model behaves the same
  as the real environment at the time the data was
  collected.
• Engineering managers can be most convinced by
  this it kind of looks like reality.
•  
• Example
• Collecting the disk activity at a Brokerage Firm
  at market open. Then running that data back
  through a real operating system.
•  
• Example
• Collecting user keystrokes used to make queries
  of a database. Then playing back those
  keystrokes.

11
Simulation
Types of Simulations

• TIME-DRIVEN SIMULATIONS
•   
• Useful for phenomena that occur at regular
  intervals.
• Example
• The instruction fetch cycle - on each clock tick
  the CPU tries to obtain an instruction ( or part
  of an instruction from memory.)
•  
• Example
• Rainfall, water usage, and the resultant height
  of a reservoir.

12
Simulation
Types of Simulations

• TIME-BASED SIMULATION CONTROL FLOW

Initialize
T T DT
Generate Events
Event 4
Event 3
Event 2
Event 1
T lt Tmax
Yes
No
13
Simulation
Types of Simulations

• EVENT-DRIVEN SIMULATIONS
•   
• Useful for phenomena that may occur at any point
  in time.
•  
• Example
• A CPU - disk system, where processes use
  variable amounts of time at each resource.
•  
• See the schematic of an event driven simulation
  below.
•  
• Example
• Cars being serviced at a tool booth. Look at
  the picture on a later page.
• Question How long would it take to write this
  simulation?

14
Simulation
Types of Simulations

• EVENT-BASED SIMULATION CONTROL FLOW

Initialize
Determine Next Event
Event Queue
T T( next_event )
case next_event
Event 1
Event 4
Event 3
Event 2
Generate New Event
Generate New Event
Generate New Event
Generate New Event
Update Statistics
15
Simulation
Types of Simulations

• SCHEMATIC OF EVENT DRIVEN SIMULATION OF A TOLL
  BOOTH

Initialize
Determine Next Event
Event Queue
T T( next_event )
case next_event
Car Approaches Line
Payment done
Put car on toll line queue if queue was empty,
generate a payment-finished event
Get next car from toll line queue if queue
empty, then get out of case.
Determine future time for next car approaching.
Determine when payment will be finished.
Generate event for "Car approaches line"
Generate event for "Payment finished"
Update Statistics
16
Simulation
Event Scheduler

• THE EVENT SCHEDULER
•  
• One of the key pieces that is provided by a
  simulation package is an event scheduler.
• It works exactly like the scheduler in a
  operating system
• accepting events that will be executed at some
  future time,
• handing out, when requested, the next event that
  will happen.
•  
• You want this scheduler to be efficient.
• It may be called on to maintain hundreds of
  upcoming events.
• To do this requires a low-cost way to insert a
  future event in a time-ordered queue.

17
Simulation
Random Number Generators

• Generating good random numbers is an art form.
  There are well known techniques for doing this
  and you are best off using the work of those who
  have gone before you.
•  
• All numbers are in fact pseudo-random. Think
  of a 16 bit random number containing a sequence
  of 65,536 numbers. Each of those numbers, when
  used as the seed of a generator produces another
  of the 65,536 numbers, and so on. They actually
  form a sequence. A good generator will produce
  all the 65,536 numbers before it repeats.
•  
• Linear Congruential Generators (LCGs) have good
  generation properties. These are of the general
  form
•  
• Xn ( a Xn-1 b ) mod m
•  
• The correct choice of a, b, and m matter a great
  deal.

18
Simulation
Random Number Generators

• /
  
• get_random_number()
• This is a method given by Jain. It
  calculates
• X ( 16807 X ) 2,147,483,647.
• However this requires 48-bit arithmetic
  (since 231 2,147,483,648)
• so the beauty of the method is that he
  reduces it to run on 32
• bits (and thus is much faster).
• 
  /
• double get_random_number( )
• long a 16807 /
  Multiplier /
• long m 2147483647 /
  Modulus /
• long q 127773 /
  m / a /
• long r 2836 /
  m mod a /
• long X_div_q
• long X_mod_q
• long static X /
  Random last-random /

19
Simulation
Random Number Generators

• if (first_time)
• while( random_seed.high gt 1000000 )
• random_seed.high - 1000000
• X (long)random_seed.high
  (long)random_seed.low
• first_time FALSE
• if ( X 0 )
• X 1
• X_div_q X / q
• X_mod_q X q
• X a X_mod_q - r X_div_q
• if ( X lt 0 )
• X X m
• return( ((double)X)/(double)m )
• / End of
  get_random_number /

20
Simulation
Random Number Generators

• TESTING RANDOM NUMBER GENERATORS
• Do random numbers produced by Excel, meet the
  properties of randomness? Lets find out!
•  
• Here are the steps
•  
• Generate 50,000 random numbers (in this case
  there were 5 columns of 10,000 numbers). Use the
  
• Take the histogram of these numbers. You see
  the result in the rightmost two columns. Picture
  Sim-1 shows the result of this.
• In Picture Sim-2 is a plot of the distribution.
  Note that the plot cheats big time!! The
  vertical scale has been adjusted by Excel to use
  the full range. But in fact the variation
  between the largest and smallest is very little.

21
Simulation
Random Number Generators

• TESTING RANDOM NUMBER GENERATORS

PICTURE SIM-1
These random numbers result from the RAND()
function.
You must first load add_ins in excel. Then the
HIST function can be found in tools?Data_Analysis?
Histogram
22
Simulation
Random Number Generators

• TESTING RANDOM NUMBER GENERATORS

PICTURE SIM-2

23
Simulation
Random Number Generators

• TESTING RANDOM NUMBER GENERATORS
• Do a chi-squared test on the numbers in the bins.
  We want to calculate the deviation of the actual
  (observed) values from the expected values. The
  formula is

1. We expect 2,500 numbers in each bin in the ideal
   case but to actually get this result would be
   highly suspicious. When we ask Excel to do the
   chi-squared distribution, we find a result of
   17.83 for the sum which according to Table A.5 is
   only 53 likely to be random. What gives? The
   conclusion we come to is that Excels Random
   Number generator is not very good.

24
Simulation
Random Number Generators

• GENERATING DISTRIBUTIONS
•  
• Weve been talking a lot about various
  distribution functions, especially Exponential
  and Poisson. How would we go about generating
  these functions?
•  
• Its real easy with Excel. You can automatically
  generate Uniform (equal probability between 0-1),
  Normal (bell curve), Bernoulli (0s and 1s
  simulating a coin toss), Binomial (number of
  successes in a given number of trials), Poisson,
  and Discrete (choose randomly from a set of
  predefined results).
• Picture SIM-3 shows the result for the Poisson
  distribution with mean 10. (Tools ? Data
  Analysis ?Random Number Generation ?Poisson).
  Then do a histogram and a graph to get the
  following picture.
• To generate an exponential function, do what Jain
  calls an inverse transformation. The inverse
  of an exponential is a ln. So Picture SIM-4 is
  the result of 50,000 instances of
  ln( rand() ).

25
Simulation
Random Number Generators

• GENERATING DISTRIBUTIONS
•  

PICTURE SIM-3
Poisson Distribution
70
60
50
40
Frequency of Count
30
20
10
0
0
5
10
15
20
25
30
-10
Count

26
Simulation
Random Number Generators

• GENERATING DISTRIBUTIONS
•  

PICTURE SIM-4
Frequency of 50,000 randomly Generated
Exponential Numbers
12000
10000
8000
Frequency
6000
4000
2000
0
0.00
0.25
0.50
0.75
1.00
1.25
1.50
1.75
2.00
2.25
2.50
2.75
3.00
3.25
3.50
3.75
4.00
4.25
4.50
4.75
5.00
Buckets of size 0.25

27
WRAPUP

• This chapter is about Simulation. Weve looked
  (at a very high level) at various ways of doing
  simulation. Weve also explored a bit about the
  mysteries of random number generators.