Performance Engineering

1
Performance Engineering
QUEUEING

• Prof. Jerry Breecher

2
WHAT WE ARE DOING HERE
We are about to embark on Queueing Lingo - all
the definitions youll ever need! Queueing -
especially the Single Queue in DETAIL Analytical
Models - how to solve a number of complex
problems using the equations we already know and
love. Works especially well for open
models. Operational Laws - drawing conclusions
about limits - what to do when we cant solve the
math exactly. Mean Value Analysis - Using the
equations in an iterative fashion to solve closed
models.
3
Queueing Models

• This section is about being able to describe the
  behavior of queues. Queues are certainly a
  prevalent object in computer systems, and our
  goal here is to write the equations that describe
  them. The language well use here is
  mathematical, but nothing really more complicated
  than algebra.

4
Queueing Lingo

• Goals
• To understand the random/statistical nature of
  computer data. We will emphasize the
  non-deterministic.
• To understand distributions for simulation
  purposes.
• To impress your friends.
• For an observation, two things matter
• The value measured.
• When measured.
• The occurrence of an event can give us the
  "when".

5
Queueing Lingo

• A STOCHASTIC PROCESS is a mechanism that produces
  a collection of measurements which all occur,
  randomly, in the same range of values. It
  applies to the VALUE measured for an observation.
  The dictionary says, "random, statistical".
• Stochastic processes are well behaved phenomena
  which don't do things which are unpredictable or
  unplanned for.
• Examples
• Throwing 2 dice always gives numbers in the range
  2 - 12.
• Actions of people are unpredictable ( unless the
  range of values is made very large.) Someone can
  always respond in a way you haven't predicted.

6
Queueing Lingo

• THE POISSON PROCESS applies to WHEN an
  observation is made. It looks random the
  arrival points are uniformly distributed across a
  time interval. Poisson processes can be defined
  by
• Event counting The distribution of the number of
  events occurring in
• a particular time is a Poisson distribution.
• Time between events The distribution of times
  between event occurrences is
• exponential.
• Example Show how a random "look" leads to an
  exponential distribution. See the next page for
  a picture of these distributions.

7
Queueing Lingo
F(t) exp(-t)

• This is a simple exponential curve. What
  properties can you identify from it?

8
Queueing Lingo
F(k) ( 5 / k! ) exp( -5 )

• Example of the Poisson Probability Density
  Function.

9
Lab - You Get To See It Happen Before Your Very
Eyes!

• This is a group lab designed to demonstrate that
  the number of random events in a particular
  interval is a Poisson distribution, and that the
  distribution of spaces between events is
  exponential.
• Open your textbook 20 times. Each time, write
  down the number of the left-hand page ( mod 100 )
  beside the appropriate sample in column I.
• Count how many of the segments in Column I have 0
  samples in them. Put this number in the first
  row of Column II. Repeat for 1, 2, 3, ...
  samples.
• For each of the samples in Column I, determine
  the distance or separation between that sample
  and the next higher sample. If the the distance
  is 0 ( the two samples are the same ) put a tick
  in the first row of column 3, if the separation
  is 3, tick the interval 3 row, etc.

10
Lab - You Get To See It Happen Before Your Very
Eyes!
Column III Number of instances of intervals
between samples.
Column II Number of segments containing N samples
Column I Numbers falling in each segment.
11
Lab - You Get To See It Happen Before Your Very
Eyes!
// Generate Poisson. // This program generates
random numbers, puts them into buckets, // and
calculates the distance between numbers. // //
Inputs // N - number of samples to
generate include lttime.hgt define
MAX_DATA_VALUE 100 define MAX_BUCKETS
100 define MAX_DATA
1000 // Compare routine used in sorting int
compare (const void a, const void b)
return ( (int)a - (int)b ) int main(
int argc, char argv) int
DataMAX_DATA int BucketMAX_BUCKETS
int NumberOfSamples int
NumberOfBuckets 10 int
LargestBucketFill 0 int Index, Temp,
i if ( argc lt2 ) printf(
"Usage GeneratePoisson ltnumber_of_samplesgt\n")
exit(0) NumberOfSamples
atoi( argv1 ) srand((unsigned)time(NULL))
for ( Index 0 Index lt NumberOfBuckets
Index ) BucketIndex 0 for (
Index 0 Index lt NumberOfSamples Index )
DataIndex rand()
MAX_DATA_VALUE Temp (NumberOfBucketsDataInde
x)/MAX_DATA_VALUE BucketTemp if (
BucketTemp gt LargestBucketFill )
LargestBucketFill BucketTemp
printf("\n Raw Random Numbers\n") for (
Index 0 Index lt NumberOfSamples Index )
printf( "d ", DataIndex )
printf("\n") qsort (Data, NumberOfSamples,
sizeof(int), compare) printf( "\n Sorted
Random Numbers\n") for ( Index 0 Index lt
NumberOfSamples Index ) printf( "d
", DataIndex ) printf("\n") printf(
"\n Range Number\n") printf( "
in Range\n") for ( Index 0 Index lt
NumberOfBuckets Index ) printf(
"2d - 2d, d\n", (MAX_DATA_VALUE/NumberOfBuc
kets Index), (MAX_DATA_VALUE/NumberOfBuck
ets (Index1)) - 1, BucketIndex ) //
Calculate distribution of items in each bucket
printf("\n") for ( Index 0 Index lt
LargestBucketFill Index ) Temp
0 for ( i 0 i ltNumberOfBuckets i )
if ( Bucketi Index ) Temp printf(
"Number of buckets with d items, d\n", Index,
Temp ) // Determine the distance
between the random items. for ( Index 0
Index lt MAX_BUCKETS Index )
BucketIndex 0 LargestBucketFill
0 for ( Index 0 Index lt NumberOfSamples -
1 Index ) Temp DataIndex1 -
DataIndex BucketTemp if ( Temp gt
LargestBucketFill ) LargestBucketFill
Temp printf( "\n Distance
Number\n") printf( " Between with
this\n") printf( " Samples
distance\n") for ( Index 0 Index lt
LargestBucketFill Index ) printf(
"2d, d\n", Index, BucketIndex )
Code is here so it doesnt get lost!!
12
Lab - You Get To See It Happen Before Your Very
Eyes!
Experimental Results
The results of running this code can be seen here
? This is the result of running 4 experiments of
20 samples each. The results arent pretty!!
Perhaps by running the tests many times,
everything would look nice and smooth. You can
see the results you would expect, but theres
lots of jitter.
13
Queueing Lingo

• Examples
• Suppose that a piece of software has an expected
  lifetime of 5 years, and that the average bug
  rate for this type of product is one bug/year.
• What is the bug expectation rate per year at the
  start of the five years, assuming this code is
  "average"?
• After two years, four bugs have been found. The
  code is still considered "average". How many
  bugs/year can be expected for the remaining three
  years?

14
Queueing Lingo

• MEMORYLESS means that the probability of an
  event doesn't depend on its past. The above case
  highlights an example where the past does matter.
• Examples Which depend on the past, and which
  don't?
• Throwing dice?
• A disk seek distance?
• The address of an instruction execution?
• The measurement of the length of a table?

Example Consider a bus stop where the time
between bus arrivals is exponentially distributed
with a rate L. Thus the bus arrivals form a
Poisson process. If you walk up to the bus stop,
how long do you have to wait until the next bus
arrives?
Prepare to Have Your Mind Bent In An Unexpected
Way!!
15
Queueing Lingo

• Example
• Consider a bus stop where the time between bus
  arrivals is exponentially distributed with a rate
  L. Thus the bus arrivals form a Poisson process.
  If you walk up to the bus stop, how long do you
  have to wait until the next bus arrives?
• 1. Possible solution Since buses arrive at a
  rate L, the average time between arrivals is 1/L.
  But since we walk up at random, we would wait
  for only half an interval on the average. So we
  would wait 1/(2L) for the next bus.
• 2. Possible solution Since the time between
  buses is exponentially distributed, it is
  memoryless. So the residual lifetime for any
  time that I arrive should be distributed
  exponentially the same way as the original
  distribution. Since the average time between
  buses is 1/L, the average time ( residual ) to
  wait should also be 1/L.

16
Queueing Lingo
This program generates 1 week's worth of bus
arrivals. We assume that the average bus arrival
rate is every 12 hours, The arrival rate is 1/12
per hour. We'll take 1 week's worth of arrivals
- we will assume that there are 14 arrivals in a
week - that's how we set the average arrival
rate. Oh - and we assume the buses arrive on the
hour (it's simpler that way). Sorted Random
Numbers 2 5 73 81 100 100 102 109 134 136 145 148
152 154

• // GenerateRandomBusArrivals
• include lttime.hgt
• define MAX_DATA 1000
• // Hours in a week
• define MAX_DATA_VALUE 168
• // Compare routine used in sorting
• int compare (const void a, const void b)
• return ( (int)a - (int)b )
• int main( int argc, char argv)
• int DataMAX_DATA
• int NumberOfSamples 14
• int Index
• printf( "This program generates 1 week's
  worth of bus arrivals.\n")
• printf( "We assume that the average bus
  arrival rate is every 12 hours,\n")
• printf( "The arrival rate is 1/12 per hour.
  We'll take 1 week's worth\n")

So given these actual bus arrivals, how long on
average must you wait for a bus?
17
Queueing Lingo

• Types of Stochastic Processes

Arrival rates are often Poisson. Service times
are often exponential. ( In other words,
they're both random. )
18
PROPERTIES OF QUEUES
Customer Arrivals
The queue A place where customers are stored
before being serviced.
Customer Departures
The device doing the actual service of the
customers.
19
PROPERTIES OF QUEUES

• How do we describe a queue? These are the
  important aspects
• Arrival process
• Service Time Distribution
• Number of Servers
• System Capacity
• Population Size
• Service Discipline
• The shorthand for queue description is thus
  A / S / m / B / K / SD.
• The inter-arrival and service times are typically
  of the following types
• M Exponential Memoryless, the distribution
  weve just been talking about.
• D Deterministic the times are constant and
  there is no variance.
• G General distribution is not specified and the
  results are valid for all distributions

20
THE SINGLE QUEUE

• If we have a single queue obeying certain
  properties, we can get all kinds of nice metrics.
  But, it must have those required properties!!
• REQUIRED PROPERTIES
• Arrivals are random with a rate of X per time.
  ( Poisson when we say this, we mean the
  inter-arrival time is exponentially distributed.
  ) Note that in steady state, throughput
  arrival rate. Many texts use l for this.
• Service times are random with a value of D.
  (Exponential ) Note this is the Demand we've
  seen before. Many texts use m for this. The
  rate of service is m 1/D.
• There's the possibility of an infinite number of
  customers.
• There's a single server. So the derivation
  were about to do doesn't work for a
  multiprocessor CPU.

21
THE SINGLE QUEUE

• These are general requirements and hold for many
  practical applications. Other analysis can be
  done for cases outside these requirements, but
  we don't do it here.
• We will apply a method called local balance to a
  system like that pictured on the next page
• The queue is of type M / M / 1.

22
THE SINGLE QUEUE
For simplification, in this particular case, the
utilization U is related to throughput and demand
by U X D
(Remember N XS) Note U l / m pi
U , p0 ( 1 U )
23
THE SINGLE QUEUE

• By Definition A queue is defined to contain
  customers that are both waiting and being
  serviced.
• In an equilibrium state, from the picture below,
  the following equations can be formed
• m pi l p i-1
• p i ( l / m ) p i-1
• p i ( l / m )i p 0 U i p 0
• The probability of having i customers in the
  queue is
• pi ( 1 U ) U i
• Note that p0 ( 1 - U ) so p i gt 0 U. But
  this is just the utilization we defined before.

24
THE SINGLE QUEUE

• The average number of customers in the queue
  (waiting and being serviced) is
• From Little's Law ( N X T ) in steady state, we
  can derive the average time spent at the queueing
  center ( both in the queue and being serviced ).
  Note what happens to this response time as the
  utilization increases!

25
THE SINGLE QUEUE

• Example
• Pat is designing a communications server that
  receives requests from "higher level" routines.
  The requests are collected by a Request Handler
  that does nothing but put them into buffers.
  These requests are removed from the buffers by
  the Request Processor on a first come first serve
  basis.
• requests -gt Request Handler -gt Buffersn -gt
  Request Processor -gt
• The requests arrive randomly at a rate of
  5/second. The Request Processor can service 10
  items per second from the buffer.
• Since allocating buffers is an expensive
  business, Pat wants to preallocate an adequate
  number of buffers so none need be allocated 99
  of the time. Clearly there's a tradeoff here
  between memory usage and time-to-allocate.
• How many buffers should be preallocated?

26
THE SINGLE QUEUE
This is the setup for the case of M / M / 2.
The probability of transition from a lower
population to a higher one is the same as before
( arrivals are the same.) But the probability of
one of the TWO servers finishing is twice as
great when both of them are filled.
X l
X l
X l
State with 0 in Queue
State with 1 in Queue
State with 2 in Queue
State with 3 in Queue
m 1/D
2m
2m
27
THE SINGLE QUEUE
The following equations hold for the single queue.
28
ANALYTICAL MODEL

• Goals
• You should be able to create, use, and understand
  a simple analytical model.
• You should have a general idea of when such
  models are applicable and should understand some
  of the buzzwords.
• PERSPECTIVE
• An Analytical Model uses mathematical operations
  to capture the relationships between observable
  quantities. The computations don't necessarily
  mimic real actions as they do for simulations.
• Examples
• The equations we've been using such as Little's
  Law.
• Local Balance Equations - these enumerate the
  states the system can be in and then determine
  the transitions between states. (This is what we
  just did with single queues.)
• Mean Value Analysis - an iterative approach using
  the equations we've already learned.

29
ANALYTICAL MODEL

• Analytical models can be applied to
• Single Queues
• Queueing Networks
• Queueing Networks are networks of queues two or
  more queues tied together. They can be
• Open - Typical of transaction processing. Jobs
  enter and leave the system being studied.
• Closed - typical of batch or terminal systems.
  Jobs always remain somewhere within the system.
• Single Class - the customers are
  indistinguishable from each other they have
  similar service demands and routing
  characteristics.
• Multiple Class - several categories of customers
  can be identified. A batch class, for example,
  might be heavily CPU bound while a terminal class
  is I/O bound. To use a multiple class model
  requires determining the characteristics for EACH
  of the classes involved.

30
ANALYTICAL MODEL
SINGLE CLASS OPEN QUEUEING NETWORK MODEL
SOLUTIONS This is EASY!! It's simply an
extension of the equations we used for the single
queue. Utilization Uk X Dk
Dk is the service time) Throughput Xk X
Vk (Throughput arrivals) Max.
Throughput Xmax Residence Time Rk
(delay centers) (queueing centers)
31
ANALYTICAL MODEL
SINGLE CLASS OPEN QUEUEING NETWORK MODEL
SOLUTIONS Queue Length Qk (delay
centers) (queueing centers) System
Response T R Time Average Number N Q
In System Remember, N Number of requests
in the "system" X Throughput R Residence time
per request. S Service Time Little's law is 90
of all you'll ever need.
We'll be using the FIGURE on the next page and
applying these Laws to the system shown there, at
a number of different levels.
32
ANALYTICAL MODEL
4
Terminals
3
CPU
2
DISK A
1
DISK C
DISK B
33
ANALYTICAL MODEL
Box 1 in the Figure A single resource, not
including the queue. Here the population, N, is
either 1 or 0 ( in use or not ). Utilization is
equal to the average number of requests present,
or the average N. Throughput X is the number of
requests serviced per time. Residence time R is,
for this case, the service time. NOTE Little's
Law reduces to Utilization Law.
DISK C
1
Example Suppose a disk, serves 100
requests/second, with the average request needing
0.008 seconds of disk service. What is the
utilization of the disk?
N X R U X S R S/(1 U )
34
ANALYTICAL MODEL
Box 2 in the Figure A single resource,
including the queue. Now the population includes
both those requests in the queue and in
service. Throughput remains the rate that the
resource satisfies requests. Residence time is
the sum of queueing and service times.
2
DISK C
Example You will need to use the result from
the previous slide. What is the time a request
spends at the disk subsystem ( spindle
queue)? What is the average number of requests
in Box 2? For one of these requests, what is
the average queueing time, and what is the
average service time?
N X R U X S R S/(1 U )
35
ANALYTICAL MODEL
Box 3 in the Figure Central Subsystem, not
including terminals. The population now is the
number of users with activity in the system -
those not "thinking". Throughput is the rate that
requests flow between system and users. Residence
time is now what we call response time.
Example The average system throughput (entering
leaving Box3) is 0.5/sec. There are an average
of 7.5 "ready" (waiting) users in the Box. What
is the average response time?
3
CPU
2
DISK A
DISK C
DISK B
36
ANALYTICAL MODEL
Box 4 in the Figure Entire system, including
terminals. The population is the total number of
users, both waiting and thinking. Throughput is
the rate that requests flow between system and
users. (same as box 3). Residence time is the
sum of response time and think time.
4
Terminals
3
Example There are 10 users with average think
time of 5 seconds, and the system has average
response time of 15 seconds. What is the
throughput?
37
ANALYTICAL MODEL
For system wide applications of Little's Law,
since time represents both thinking and waiting
for a response, NSS X R for Subsystem or
lower. NES X ( R Z ) for the Entire
system. Example A System has 64 interactive
users, with an average think time of 30 seconds.
Two interactions complete on average each
second. What is the average response time for
this system? How many interactions are in the
system at any time? What sized System ( how many
CPU's ) would be needed for this application?
RESPONSE TIME LAW R N/X - Z
38
ANALYTICAL MODEL
Example Problems You Should Now Be Able To
Do The average delay experienced by a packet
when traversing a computer network is 100 msec.
The average number of packets that cross the
network is 128 packets/sec. What is the average
number of packets in transit in the network?
Example Problems You Should Now Be Able To
Do Measurements taken during one hour from a Web
server indicate that the utilization of the CPU
and the two disks are UCPU 0.25, Udisk1
0.35, and Udisk2 0.30. The Web server log shows
that 21,600 requests were processed during the
measurement interval. What are the service
demands (the time used by each request) at the
CPU and both disks? What is the maximum
throughput, and what was the response time of
the Web server during the measurement interval?
Example Problems You Should Now Be Able To Do A
computer system is measured for 30 minutes.
During this time, 5,400 transactions are
completed and 18,900 I/O operations are executed
on a certain disk that is 40 utilized. What is
the average number of I/O operations per
transaction on this disk? What is the average
service time per transaction on this disk?
Solutions on a later page
39
ANALYTICAL MODEL
Example Problems You Should Now Be Able To Do A
file server is monitored for 60 minutes, during
which time 7,200 requests are completed. The disk
utilization is measured to be 30. The average
service time at this disk is 30 msec per
IO. What is the average number of accesses to
this disk per file request?
Example Problems You Should Now Be Able To Do A
computer system has one CPU and two disks disk 1
and disk 2. The system is monitored for one hour
and the utilization of the CPU and of disk 1 are
measured to be 32 and 60, respectively. Each
transaction makes 5 I/O requests to disk 1 and 8
to disk 2. The average service time at disk 1 is
30 msec and at disk 2 is 25 msec. Find the
system throughput. Find the utilization of disk
2. Find the average service demands at the CPU,
disk 1, and disk 2.
Example Problems You Should Now Be Able To Do An
interactive system has 50 terminals and the
user's think time is equal to 5 seconds. The
utilization of one of the system's disk was
measured to be 60. The average service time at
the disk is equal to 30 msec. Each user
interaction requires, on average, 4 I/Os on this
disk. What is the average response time of the
interactive system?
Solutions on a later page
40
ANALYTICAL MODEL
Problem Solution The average delay experienced
by a packet when traversing a computer network is
100 msec. The average number of packets that
cross the network is 128 packets/sec. What is
the average number of packets in transit in the
network? Straight usage of Littles Law N
XS 128 packets/sec 0.1 sec 12.8 packets
Problem Solution Measurements taken during one
hour from a Web server indicate that the
utilization of the CPU and the two disks are
UCPU 0.25, Udisk1 0.35, and Udisk2 0.30.
The Web server log shows that 21,600 requests
were processed during the measurement interval.
What are the service demands (the time used by
each request) at the CPU and both disks? What is
the maximum throughput, and what was the
response time of the Web server during the
measurement interval? There are 21,600
requests/hour 60 requests/sec. During each
second, the CPU is used 250 milliseconds so
each of the 60 requests is using 4.16
milliseconds. Similarly the disks use 5.83
milliseconds and 5 milliseconds of service per
transaction. Maximum throughput is determined by
the device having the highest utilization, Udisk1
0.35. When that disk is maxd out, there will
be 1 / 0.35 more traffic or 2.86 more. Thus
the maximum throughput will be 2.86 60
transactions/second 171 transactions/second. You
can determine the total response time by
calculating the response time at each queueing
center. This is 4.16 msec/(1 0.35) 5.83
msec / ( 1 0.35) 5.0 msec / ( 1- 0.3)
6.4 8.97 7.1 22.5 msec
41
ANALYTICAL MODEL
Problem Solution A computer system is measured
for 30 minutes. During this time, 5,400
transactions are completed and 18,900 I/O
operations are executed on a certain disk that is
40 utilized. What is the average number of I/O
operations per transaction on this disk? What is
the average service time per transaction on this
disk? Put everything into the same time units
(seconds usually work best) ? 5,400 transactions
/ 30 minutes 3 transactions/second 18,900 IO /
30 minutes 10.5 IOs / second So the number of
IOs/transaction 10.5 / 3 3.5 IOs /
transaction In each second, this disk is busy 400
milliseoncds of time. During a second, 3
transactions complete. So the disk service time
per transaction is 133 milliseconds.
Problem Solution A file server is monitored for
60 minutes, during which time 7,200 requests are
completed. The disk utilization is measured to be
30. The average service time at this disk is 30
msec per IO. What is the average number of
accesses to this disk per file request? The idea
is that a file server request may result in
multiple IOs to the disk its not a 1 to 1
match necessarily. 7,200 requests / 3,600
seconds 2 requests/second. The disk is 30
busy so it runs for 300 milliseconds each second.
The service time is 30 milliseconds, so 10 IOs
are completed each second. 10 IOs for 2 requests
means there are on average 5 IOs / request.
42
ANALYTICAL MODEL
Problem Solution A computer system has one CPU
and two disks disk 1 and disk 2. The system is
monitored for one hour and the utilization of the
CPU and of disk 1 are measured to be 32 and 60,
respectively. Each transaction makes 5 I/O
requests to disk 1 and 8 to disk 2. The average
service time at disk 1 is 30 msec and at disk 2
is 25 msec. Find the system throughput. Find
the utilization of disk 2. Find the average
service demands at the CPU, disk 1, and disk
2. It turns out here that disk 1 is where we have
the most information For this disk, its busy
600 milliseconds out of each second it takes 30
milliseconds for each IO so there are 20
IOs/second. Since each transaction makes 5 IO
requests to disk 1, that means there are 4
transactions/second. Once we have this answer,
we know there are 32 IOs to disk 2 per second.
Each of those IOs takes 25 milliseconds, giving a
total time usage of 800 milliseconds. That means
this disk is 80 utilized.
Problem Solution An interactive system has 50
terminals and the user's think time is equal to 5
seconds. The utilization of one of the system's
disk was measured to be 60. The average service
time at the disk is equal to 30 msec. Each user
interaction requires, on average, 4 I/Os on this
disk. What is the average response time of the
interactive system? The disk is doing 20 IOs /
second. Each transaction is 4 IOs, so the
throughput is 5 trans/sec. To get this throughput
requires each of the 50 users is executing a
transaction every 10 seconds. Thus the time in
the system must be 5 seconds (because the user is
already thinking for 5 secs.)
43
ANALYTICAL MODEL
FORCED FLOW LAW The Forced Flow Law states that
the flow in all parts of a system must be
consistent. Suppose we count both system
completions and also completions at each
resource. The visit count ( visit ratio ) is
defined as Resource Completion Ck System
Completion C Visit Count Vk Ck / C
THE FORCED FLOW LAW Xk Vk X
44
ANALYTICAL MODEL
Example Suppose a system has 30 terminals ( N
30 ). 18 seconds average think time ( Z 18
). 20 visits to a specific disk/interaction
(Vdisk 20 ). 30 utilization of that disk
(Udisk 0.30 ). 25 millisecs is the average
service required per visit to the disk (Sdisk
0.025 sec.). We want to know Disk throughput
System throughput Response time
THE FORCED FLOW LAW Xk Vk X
RESPONSE TIME LAW R N/X - Z
45
ANALYTICAL MODEL
Example Consider the problem of a spy from
Burger King trying to figure out how many people
are at a McDonald's. The spy can't sit inside
and watch all day, so must somehow calculate the
number from information obtained from outside
observations. Thirty customers/hour arrive on
the average ( over a long period of time ) and
the average customer exits after 12
minutes. Assuming that all this time is spent
standing in line, what is the mean queue length
in the restaurant? If the Standard Deviation of
the 30 customers is 3, what is the uncertainty of
the queue length? What happens if both the
arrival rate and the service time have
uncertainties?
THE FORCED FLOW LAW Xk Vk X
RESPONSE TIME LAW R N/X - Z
46
ANALYTICAL MODEL
EXAMPLE OF THE SOLUTION OF AN OPEN MODEL Model
Inputs Vcpu 121 Vdisk1 70 Vdisk2 50 Scpu
0.005 Sdisk1 0.030 Sdisk2 0.027 Dcpu
0.605 Ddisk1 2.1 Ddisk2 1.35 c 0.3
jobs/sec
47
Solution Of An Open Model
EXAMPLE
Model Inputs Vcpu 121 Vdisk1 70
Vdisk2 50 Scpu 0.005 Sdisk1 0.030
Sdisk2 0.027 Dcpu 0.605 Ddisk1 2.1
Ddisk2 1.35 c 0.3 jobs/sec
Model Outputs l 1 / Dmax 1 / 2.1
0.476 jobs/sec Xcpu (0.3) c Vcpu
(0.3)(121) 36.3 visits/sec Ucpu (0.3)
c Dcpu (0.3)(0.605) 0.182 Rcpu
(0.3) Dcpu / ( 1 Ucpu(0.3) ) 0.605 /
0.818 0.740 secs Qcpu (0.3) Ucpu(0.3) / ( 1
Ucpu(0.3) ) 0.182 / 0.818 0.222
jobs R(0.3) Rcpu(0.3) Rdisk1(0.3)
Rdisk2(0.3) 0.740 5.676 2.269 8.685
secs Q(0.3) l R(l) (0.3) (8.685)
2.606 jobs
48
SUMMARY OF PERFORMANCE METRICS
T is the length of TIME we observed the
system. A is the number of request ARRIVALS
observed. C is the number of request DEPARTURES
observed. W is the ACCUMULATED TIME for all
requests within the system time spent both
waiting for and using resources. B is the
length of time that the resource was observed to
be BUSY. Z is the think time of a terminal user.
Vk the visit ratio, is the number of times
device k is visited per transaction.
Arrival Rate Y A / T Throughput (Departure
Rate) X C / T Utilization U B / T Service
Requirement S B / C Requests in system N W
/ T Residence time R W / C
UTILIZATION LAW U X S LITTLE'S LAW
N X R RESPONSE TIME LAW R N/X
Z THE FORCED FLOW LAW Xk Vk X
49
OPERATIONAL LAWS
Goals To increase facility using performance
metrics. To be able to calculate limits or
boundaries on performance metrics. To be able to
do "back of the envelope" calculations.
UTILIZATION LAW U X S LITTLE'S LAW
N X R RESPONSE TIME LAW R N/X
Z THE FORCED FLOW LAW Xk Vk X
50
OPERATIONAL LAWS

• BOUNDARY VALUES
• The goal here is to make estimations of the
  outside limits of a performance parameter. We do
  this by selective "blind" application of our
  simple laws to complex systems our results give
  upper and lower bounds, not an exact answer.
• Example
• An editor program wants to read 100 disk pages
  into memory. The resources required for each
  disk read are
• 5 milliseconds of CPU Processor time.
• 4 milliseconds of Controller Processor time.
• 2 milliseconds of SCSI Handshaking time.
• 10 milliseconds of DISK seek/rotation/transfer
  time.
• What is the bottlenecking device? Warning!!
  This is a trick question!
• What is the worst throughput for this system,
  assuming the editor single threads the reads -
  waiting until each read completes before starting
  the next?
• What is the best throughput for this system? How
  long will it take to read in the 100 disk blocks?

51
OPERATIONAL LAWS

• BOUNDARY VALUES
• Now that we know the limits for one user, let's
  expand this for N users.
• Example
• Multiple users, each with their own copy of the
  editor, are reading files from the same disk
  (parameters are the same as the last example.)
• What is the bottleneck device in this case?
• Considering only the bottleneck device, what is
  the best throughput achievable?
• Now taking into account all devices, what is the
  BEST throughput N users can achieve ( assuming
  the requests NEVER get in each other's way?)
• Now taking into account all devices, what is the
  WORST throughput N users can achieve ( assuming
  the requests ALWAYS get in each other's way?)

52
OPERATIONAL LAWS
BOUNDARY VALUES We define DEMAND as the amount
of a resource used for a transaction this is
just the number of visits to that resource during
one transaction, and the service time used at
each of those visits Dk Vk Sk Dtotal
D SDk Dmax Dbottleneck SUMMARY
Best throughput overall X 1 /
DBottleneck Best throughput for N users X
N / D Worst throughput for N users X N /
( N D ) 1 / D
53
OPERATIONAL LAWS
Exercise Consider an interactive system with a
CPU and two disks. The following measurement
data was obtained by measuring the system
----------- Observation Data ---------------- Obs
ervation interval 30 minutes Active
terminals 30 Think time 12 seconds Completed
transactions 1,600 Fast disk accesses 32,000 Slow
disk accesses 12,000 CPU busy 1,080 seconds Fast
disk busy 400 seconds Slow disk busy 600 seconds
Determine the visit counts Vk, service times per
visit Sk, and service demands Dk at each center.
Data At Centers
54
OPERATIONAL LAWS
Exercise
----------- Observation Data ---------------- Obs
ervation interval 30 minutes Active
terminals 30 Think time 12 seconds Completed
transactions 1,600 Fast disk accesses 32,000 Slow
disk accesses 12,000 CPU busy 1,080 seconds Fast
disk busy 400 seconds Slow disk busy 600 seconds
Give optimistic and pessimistic asymptotic bounds
on throughput and response time for 10 and 40
active terminals.
Bounds on Throughput and Response Time
55
OPERATIONAL LAWS
Exercise
----------- Observation Data ---------------- Obs
ervation interval 30 minutes Active
terminals 30 Think time 12 seconds Completed
transactions 1,600 Fast disk accesses 32,000 Slow
disk accesses 12,000 CPU busy 1,080 seconds Fast
disk busy 400 seconds Slow disk busy 600 seconds

• Consider the following changes to the system
• Move all files to the fast disk.
• Replace the slow disk by a second fast disk.
• Increase the CPU speed by 50 (with the original
  disks.)
• Increase the CPU speed by 50 and balance the
  disk load across 2 fast disks.
• Rank the efficacy of these changes.

What changes should be made so response time will
not exceed 10 seconds?
56
MEAN VALUE ANALYSIS

• This is an extension of the Operational Analysis
  we did before. This is the method that should be
  used on closed systems.
• Assumptions
• Applies to various service disciplines and
  service time distributions. (We will only do
  fixed capacity centers here.)
• The arrival rate is not dependent on the load.
• Arrivals are a Poisson process.
• We will use a technique called Mean Value
  Analysis that is based on the repetitive
  application of three equations
• 1. Little's Law applied to the whole network
• 2. Little's Law applied to each service center
• 3. The service center residence time equations

A great book on this, now out of print is
at http//www.cs.washington.edu/homes/lazowska/qsp
/
Menasce, et. al. do this in Section 12.3 of the
text.
57
MEAN VALUE ANALYSIS
1. Little's Law applied to the whole
network where X(N) is the system throughput,
Z is the think time, and Rk (N) the residence
time at center k when there are N customers in
the network. 2. Little's Law applied to each
service center 3. The service center
residence time equations Delay
Centers Queueing Centers where Ak (N) is
the average number of customers seen at center k
when a new customer arrives. The number seen on
arrival (in equilibrium) is equal to the average
queue length when there is one less customer in
the system! So the equations bootstrap!!!
58
MEAN VALUE ANALYSIS
//////////////////////////////////////////////////
////////////////////////////// // This program
does very simple mean value analysis. ////////////
//////////////////////////////////////////////////
////////////////// define MAX_NUMBER_OF_CENTER
S 30 int number_of_customers int
number_of_centers / of centers
requested by the user. / int
number_of_customers / The population
in each class / float input double
think_time / The time spent
at the terminal / double demandMAX_NUMBER_OF_
CENTERS/ Service demand of a center. / double
q_lengthMAX_NUMBER_OF_CENTERS double
residence_timeMAX_NUMBER_OF_CENTERS1 //
Residence time at a center. double
throughput double system_response_time //
Prototypes void print_the_numbers(int n)
This program is at http//web.cs.wpi.edu/jb/pe
rf/Lectures/mva.c
59
MEAN VALUE ANALYSIS
main() char input_string80
int center, n ////////////////////
//////////////////////////////////////////////////
////// // Get the input information from the
user. ////////////////////////////////////////
//////////////////////////////////// printf(
"Number of users ") gets( input_string )
number_of_customers atoi( input_string )
printf( "Number of centers ") gets(
input_string ) number_of_centers atoi(
input_string ) printf( "Think Time ")
gets( input_string ) sscanf( input_string,
"f", input ) think_time (double)input
for ( center 0 center lt number_of_centers
center ) printf( " Demand
for center d ", center) gets( input_string
) sscanf( input_string, "f", input
) demandcenter (double)input
////////////////////////////////////////////
//////////////////////////////// // Now that
the network is described, we perform the
evaluation. // Begin by initializing
to the trivial solution for 0 customers.
//////////////////////////////////////////////////
////////////////////////// for ( center 0
center lt number_of_centers center )
q_lengthcenter 0
60
MEAN VALUE ANALYSIS
////////////////////////////////////////////////
//////////////////////////// // The
algorithm now iterates through each population
value n. /////////////////////////////////////
/////////////////////////////////////// for (
n 0 n lt number_of_customers n )
system_response_time 0 for ( center
0 center lt number_of_centers center )
residence_timecenter
demandcenter ( 1.0 q_lengthcenter )
system_response_time
residence_timecenter
//////////////////////////////////////////////////
////////////////////////// // Next use
Little's Law to compute the system throughput.
/////////////////////////////////////////////////
/////////////////////////// throughput
n / ( think_time system_response_time )
//////////////////////////////////////////////////
////////////////////////// // Finally use
Little's Law to compute center queue lengths.
//////////////////////////////////////////////////
////////////////////////// for ( center
0 center lt number_of_centers center )
q_lengthcenter
residence_timecenter throughput
print_the_numbers(n)
/ END OF LOOP THROUGH POPULATION /
exit(0) / END OF MAIN /
61
MEAN VALUE ANALYSIS
//////////////////////////////////////////////////
////////////////////////////// // Print the
results of each loop /////////////////////////////
//////////////////////////////////////////////////
/ void print_the_numbers(int n) int
i printf( "\n" ) printf( "Iteration
d System Response Time f Throughput
f\n", n,
system_response_time, throughput ) for ( i
0 i lt number_of_centers i ) printf(
"Center d Residence Time f Queue
Length f\n", i,
residence_timei, q_lengthi ) //
END OF PRINT_THE_NUMBERS
62
MEAN VALUE ANALYSIS
EXAMPLE OF THE SOLUTION OF A CLOSED MODEL Model
Inputs Vcpu 121 Vdisk1 70 Vdisk2 50 Scpu
0.005 Sdisk1 0.030 Sdisk2 0.027 Dcpu
0.605 Ddisk1 2.1 Ddisk2 1.35 c 0.3
jobs/sec
The INPUT numbers are the same as were used for
the open model. We modify the picture so there
are no arrivals or departures, but instead
there are terminals the jobs stay in the
picture (which is what makes it closed.) There
are 3 terminal users with average think time of
15 seconds N 3, Z 15.
63
Solution Of An Open Model
EXAMPLE
Model Inputs Vcpu 121 Vdisk1 70
Vdisk2 50 Scpu 0.005 Sdisk1 0.030
Sdisk2 0.027 Dcpu 0.605 Ddisk1 2.1
Ddisk2 1.35 c 0.3 jobs/sec
To start with, we initialize the queue lengths to
0 Qcpu Qdisk1 Qdisk2 0 Iteration
Number 1 Rcpu 0.605 sec Rdisk1 2.1
sec Rdisk2 1.35 sec. X 1 / ( 15 4.055 )
0.0525 Qcpu 0.0318 Qdisk1 0.1102 Qdisk2
0.0708 Iteration Number 2 Rcpu 0.624
sec Rdisk1 2.331 sec Rdisk2 1.446 sec. X
0.1030 Qcpu 0.0643 Qdisk1 0.2403 Qdisk2
0.1490
64
Solution Of An Open Model
EXAMPLE
Model Inputs Vcpu 121 Vdisk1 70
Vdisk2 50 Scpu 0.005 Sdisk1 0.030
Sdisk2 0.027 Dcpu 0.605 Ddisk1 2.1
Ddisk2 1.35 c 0.3 jobs/sec
Iteration Number 3 Rcpu 0.644 sec Rdisk1
2.605 sec Rdisk2 1.551 sec. X 0.1515 Qcpu
0.0976 Qdisk1 0.3947 Qdisk2 0.2350 From
these numbers we can calculate X(3)
0.1515 R(3) ( N / X ) - Z 4.74 Q(3) N
- X(3) Z 0.72 Theres a very nice example
on Jain, pp 578-9.
65
CONCLUSION
This section has discussed Queueing Lingo -
all the definitions youll ever need! Queueing
- especially the Single Queue in
DETAIL Analytical Models - how to solve a
number of complex problems using the equations we
already know and love. Works especially well for
open models. Operational Laws - drawing
conclusions about limits - what to do when we
cant solve the math exactly. Mean Value
Analysis - Using the equations in an iterative
fashion to solve closed models.