Queues Single and Multiple

1
QueuesSingle and Multiple
We look at the description of single queues and
see how it applies in a network of queues.
2
WHAT WE ARE DOING HERE
We are about to embark on Queueing Lingo - all
the definitions youll ever need! Queueing -
especially the Single Queue in DETAIL Analytical
Models - how to solve a number of complex
problems using the equations we already know and
love. Works especially well for open
models. The material in these slides can be
found in any standard Performance Analysis
text.
3
Queueing Lingo

• Goals
• To understand the random/statistical nature of
  computer data. We will emphasize the
  non-deterministic.
• To understand distributions for simulation
  purposes.
• To impress your friends.
• For an observation, two things matter
• The value measured.
• When measured.
• The occurrence of an event can give us the
  "when".

4
Queueing Lingo

• A STOCHASTIC PROCESS is a mechanism that produces
  a collection of measurements which all occur,
  randomly, in the same range of values. It
  applies to the VALUE measured for an observation.
  The dictionary says, "random, statistical".
• Stochastic processes are well behaved phenomena
  which don't do things which are unpredictable or
  unplanned for.
• Examples
• Throwing 2 dice always gives numbers in the range
  2 - 12.
• Actions of people are unpredictable ( unless the
  range of values is made very large.) Someone can
  always respond in a way you haven't predicted.

5
PROPERTIES OF QUEUES
Customer Arrivals
The queue A place where customers are stored
before being serviced.
Customer Departures
The device doing the actual service of the
customers.
6
PROPERTIES OF QUEUES

• How do we describe a queue? These are the
  important aspects
• Arrival process
• Service Time Distribution
• Number of Servers
• System Capacity
• Population Size
• Service Discipline
• The shorthand for queue description is thus
  A / S / m / B / K / SD.
• The inter-arrival and service times are typically
  of the following types
• M Exponential Memoryless, the distribution
  weve just been talking about.
• D Deterministic the times are constant and
  there is no variance.
• G General distribution is not specified and the
  results are valid for all distributions

7
THE SINGLE QUEUE

• If we have a single queue obeying certain
  properties, we can get all kinds of nice metrics.
  But, it must have those required properties!!
• REQUIRED PROPERTIES
• Arrivals are random with a rate of X per time.
  ( Poisson when we say this, we mean the
  inter-arrival time is exponentially distributed.
  ) Note that in steady state, throughput
  arrival rate. Many texts use l for this.
• Service times are random with a value of D.
  (Exponential ) Note this is the Demand we've
  seen before. Many texts use m for this. The
  rate of service is m 1/D.
• There's the possibility of an infinite number of
  customers.
• There's a single server. So the derivation
  were about to do doesn't work for a
  multiprocessor CPU.

8
THE SINGLE QUEUE

• These are general requirements and hold for many
  practical applications. Other analysis can be
  done for cases outside these requirements, but
  we don't do it here.
• We will apply a method called local balance to a
  system like that pictured on the next page
• The queue is of type M / M / 1.

9
THE SINGLE QUEUE
For simplification, in this particular case, the
utilization U is related to throughput and demand
by U X D
(Remember N XS Littles Law) Note U
l / m pi U , p0 ( 1 U )
10
THE SINGLE QUEUE

• By Definition A queue is defined to contain
  customers that are both waiting and being
  serviced.
• In an equilibrium state, from the picture below,
  the following equations can be formed
• m pi l p i-1
• p i ( l / m ) p i-1
• p i ( l / m )i p 0 U i p 0
• The probability of having i customers in the
  queue is
• pi ( 1 U ) U i
• Note that p0 ( 1 - U ) so p i gt 0 U. But
  this is just the utilization we defined before.

11
THE SINGLE QUEUE

• The average number of customers in the queue
  (waiting and being serviced) is
• From Little's Law ( N X T ) in steady state, we
  can derive the average time spent at the queueing
  center ( both in the queue and being serviced ).
  Note what happens to this response time as the
  utilization increases!

12
THE SINGLE QUEUE
This is the setup for the case of M / M / 2.
The probability of transition from a lower
population to a higher one is the same as before
( arrivals are the same.) But the probability of
one of the TWO servers finishing is twice as
great when both of them are filled.
X l
X l
X l
State with 0 in Queue
State with 1 in Queue
State with 2 in Queue
State with 3 in Queue
m 1/D
2m
2m
13
THE SINGLE QUEUE
The following equations hold for the single queue.
14
ANALYTICAL MODEL

• Goals
• To create, use, and understand a simple
  analytical model.
• To have a general idea of when such models are
  applicable.
• PERSPECTIVE
• An Analytical Model uses mathematical operations
  to capture the relationships between observable
  quantities. The computations don't necessarily
  mimic real actions as they do for simulations.
• Examples
• The equations we've been using such as Little's
  Law.
• Local Balance Equations - these enumerate the
  states the system can be in and then determine
  the transitions between states. (This is what we
  just did with single queues.)
• Mean Value Analysis - an iterative approach using
  the equations we've already learned.

15
ANALYTICAL MODEL

• Analytical models can be applied to
• Single Queues
• Queueing Networks
• Queueing Networks are networks of queues two or
  more queues tied together. They can be
• Open - Typical of transaction processing. Jobs
  enter and leave the system being studied.
• Closed - typical of batch or terminal systems.
  Jobs always remain somewhere within the system.
• Single Class - the customers are
  indistinguishable from each other they have
  similar service demands and routing
  characteristics.
• Multiple Class - several categories of customers
  can be identified. A batch class, for example,
  might be heavily CPU bound while a terminal class
  is I/O bound. To use a multiple class model
  requires determining the characteristics for EACH
  of the classes involved.

16
ANALYTICAL MODEL
SINGLE CLASS OPEN QUEUEING NETWORK MODEL
SOLUTIONS This is EASY!! It's simply an
extension of the equations we used for the single
queue. Utilization Uk X Dk
Dk is the service time) Throughput Xk X
Vk (Throughput arrivals) Max.
Throughput Xmax Residence Time Rk
(delay centers) (queueing centers)
17
ANALYTICAL MODEL
SINGLE CLASS OPEN QUEUEING NETWORK MODEL
SOLUTIONS Queue Length Qk (delay
centers) (queueing centers) System
Response T R Time Average Number N Q
In System Remember, N Number of requests
in the "system" X Throughput R Residence time
per request. S Service Time Little's law is 90
of all you'll ever need.
We'll be using the FIGURE on the next page and
applying these Laws to the system shown there, at
a number of different levels.
18
ANALYTICAL MODEL
4
Terminals
3
CPU
2
DISK A
1
DISK C
DISK B
19
ANALYTICAL MODEL
Box 1 in the Figure A single resource, not
including the queue. Here the population, N, is
either 1 or 0 ( in use or not ). Utilization is
equal to the average number of requests present,
or the average N. Throughput X is the number of
requests serviced per time. Residence time R is,
for this case, the service time. NOTE Little's
Law reduces to Utilization Law.
DISK C
1
Example Suppose a disk, serves 100
requests/second, with the average request needing
0.008 seconds of disk service. What is the
utilization of the disk?
N X R U X S R S/(1 U )
20
ANALYTICAL MODEL
Box 2 in the Figure A single resource,
including the queue. Now the population includes
both those requests in the queue and in
service. Throughput remains the rate that the
resource satisfies requests. Residence time is
the sum of queueing and service times.
2
DISK C
Example You will need to use the result from
the previous slide. What is the time a request
spends at the disk subsystem ( spindle
queue)? What is the average number of requests
in Box 2? For one of these requests, what is
the average queueing time, and what is the
average service time?
N X R U X S R S/(1 U )
21
ANALYTICAL MODEL
Box 3 in the Figure Central Subsystem, not
including terminals. The population now is the
number of users with activity in the system -
those not "thinking". Throughput is the rate that
requests flow between system and users. Residence
time is now what we call response time.
Example The average system throughput (entering
leaving Box3) is 0.5/sec. There are an average
of 7.5 "ready" (waiting) users in the Box. What
is the average response time?
3
CPU
2
DISK A
DISK C
DISK B
22
ANALYTICAL MODEL
Box 4 in the Figure Entire system, including
terminals. The population is the total number of
users, both waiting and thinking. Throughput is
the rate that requests flow between system and
users. (same as box 3). Residence time is the
sum of response time and think time.
4
Terminals
3
Example There are 10 users with average think
time of 5 seconds, and the system has average
response time of 15 seconds. What is the
throughput?
23
ANALYTICAL MODEL
For system wide applications of Little's Law,
since time represents both thinking and waiting
for a response, NSS X R for Subsystem or
lower. NES X ( R Z ) for the Entire
system. Example A System has 64 interactive
users, with an average think time of 30 seconds.
Two interactions complete on average each
second. What is the average response time for
this system? How many interactions are in the
system at any time? What sized System ( how many
CPU's ) would be needed for this application?
RESPONSE TIME LAW R N/X - Z
24
ANALYTICAL MODEL
Example Problems You Should Now Be Able To
Do The average delay experienced by a packet
when traversing a computer network is 100 msec.
The average number of packets that cross the
network is 128 packets/sec. What is the average
number of packets in transit in the network?
Example Problems You Should Now Be Able To
Do Measurements taken during one hour from a Web
server indicate that the utilization of the CPU
and the two disks are UCPU 0.25, Udisk1
0.35, and Udisk2 0.30. The Web server log shows
that 21,600 requests were processed during the
measurement interval. What are the service
demands (the time used by each request) at the
CPU and both disks? What is the maximum
throughput, and what was the response time of
the Web server during the measurement interval?
Example Problems You Should Now Be Able To Do A
computer system is measured for 30 minutes.
During this time, 5,400 transactions are
completed and 18,900 I/O operations are executed
on a certain disk that is 40 utilized. What is
the average number of I/O operations per
transaction on this disk? What is the average
service time per transaction on this disk?
25
ANALYTICAL MODEL
Example Problems You Should Now Be Able To Do A
file server is monitored for 60 minutes, during
which time 7,200 requests are completed. The disk
utilization is measured to be 30. The average
service time at this disk is 30 msec per file
operation request. What is the average number
of accesses to this disk per file request?
Example Problems You Should Now Be Able To Do A
computer system has one CPU and two disks disk 1
and disk 2. The system is monitored for one hour
and the utilization of the CPU and of disk 1 are
measured to be 32 and 60, respectively. Each
transaction makes 5 I/O requests to disk 1 and 8
to disk 2. The average service time at disk 1 is
30 msec and at disk 2 is 25 msec. Find the
system throughput. Find the utilization of disk
2. Find the average service demands at the CPU,
disk 1, and disk 2.
Example Problems You Should Now Be Able To Do An
interactive system has 50 terminals and the
user's think time is equal to 5 seconds. The
utilization of one of the system's disk was
measured to be 60. The average service time at
the disk is equal to 30 msec. Each user
interaction requires, on average, 4 I/Os on this
disk. What is the average response time of the
interactive system?
26
ANALYTICAL MODEL
FORCED FLOW LAW The Forced Flow Law states that
the flow in all parts of a system must be
consistent. Suppose we count both system
completions and also completions at each
resource. The visit count ( visit ratio ) is
defined as Resource Completion Ck System
Completion C Visit Count Vk Ck / C
THE FORCED FLOW LAW Xk Vk X
27
ANALYTICAL MODEL
Example Suppose a system has 30 terminals ( N
30 ). 18 seconds average think time ( Z 18
). 20 visits to a specific disk/interaction
(Vdisk 20 ). 30 utilization of that disk
(Udisk 0.30 ). 25 millisecs is the average
service required per visit to the disk (Sdisk
0.025 sec.). We want to know Disk throughput
System throughput Response time
THE FORCED FLOW LAW Xk Vk X
RESPONSE TIME LAW R N/X - Z
28
ANALYTICAL MODEL
Example Consider the problem of a spy from
Burger King trying to figure out how many people
are at a McDonald's. The spy can't sit inside
and watch all day, so must somehow calculate the
number from information obtained from outside
observations. Thirty customers/hour arrive on
the average ( over a long period of time ) and
the average customer exits after 12
minutes. Assuming that all this time is spent
standing in line, what is the mean queue length
in the restaurant?
THE FORCED FLOW LAW Xk Vk X
RESPONSE TIME LAW R N/X - Z
29
ANALYTICAL MODEL
EXAMPLE OF THE SOLUTION OF AN OPEN MODEL Model
Inputs Vcpu 121 Vdisk1 70 Vdisk2 50 Scpu
0.005 Sdisk1 0.030 Sdisk2 0.027 Dcpu
0.605 Ddisk1 2.1 Ddisk2 1.35 c 0.3
jobs/sec
30
Solution Of An Open Model
EXAMPLE
Model Inputs Vcpu 121 Vdisk1 70
Vdisk2 50 Scpu 0.005 Sdisk1 0.030
Sdisk2 0.027 Dcpu 0.605 Ddisk1 2.1
Ddisk2 1.35 c 0.3 jobs/sec
Model Outputs l 1 / Dmax 1 / 2.1
0.476 jobs/sec Xcpu (0.3) c Vcpu
(0.3)(121) 36.3 visits/sec Ucpu (0.3)
c Dcpu (0.3)(0.605) 0.182 Rcpu
(0.3) Dcpu / ( 1 Ucpu(0.3) ) 0.605 /
0.818 0.740 secs Qcpu (0.3) Ucpu(0.3) / ( 1
Ucpu(0.3) ) 0.182 / 0.818 0.222
jobs R(0.3) Rcpu(0.3) Rdisk1(0.3)
Rdisk2(0.3) 0.740 5.676 2.269 8.685
secs Q(0.3) l R(l) (0.3) (8.685)
2.606 jobs
31
SUMMARY OF PERFORMANCE METRICS
T is the length of TIME we observed the
system. A is the number of request ARRIVALS
observed. C is the number of request DEPARTURES
observed. W is the ACCUMULATED TIME for all
requests within the system time spent both
waiting for and using resources. B is the
length of time that the resource was observed to
be BUSY. Z is the think time of a terminal user.
Vk the visit ratio, is the number of times
device k is visited per transaction.
Arrival Rate Y A / T Throughput (Departure
Rate) X C / T Utilization U B / T Service
Requirement S B / C Requests in system N W
/ T Residence time R W / C
UTILIZATION LAW U X S LITTLE'S LAW
N X R RESPONSE TIME LAW R N/X
Z THE FORCED FLOW LAW Xk Vk X