Queueing Systems

1
Queueing Systems
Basic Structure

• Note
• Customers need not be people ? parts, vehicles,
  machines, jobs.
• Queue might not be a physical line ? customers on
  hold, jobs waiting to be printed, planes circling
  airport.

2
Components of Model
Input Source The size of the calling
population may be modeled as
infinite or finite. Calculations are
easier in the infinite case and in many cases
this is a reasonable approximation (bank,
pizza parlor, blood bank).
Queueing Discipline First-come first-served
(FIFO) is most frequent assumption, but
priority servicing is important in some
settings.
Service Mechanism One or more servers may be
placed in parallel.
3
Typical Performance Questions
What is the ... 1. average number of
customers in system? 2. average time a
customer spends in system? 3. probability a
customer is rejected? 4. fraction of time a
server is idle?
These questions are aimed at characterizing
complex systems. Analyses used to support
decision-making. In queueing ( most analyses of
complex stochastic systems), OR takes the form
of asking what if questions rather than
trying to optimize the design.
4
Notation and Terminology

N(t) of customers in the system at time t
? 0 ?n(t) probability exactly n customers in
system at time t, given in system at
time 0 s of parallel servers ?n mean
arrival rate (expected of arrivals per unit
time) ?n mean service rate (expected of
departures per unit time) (Both ?n and ?n assume
n customers are in system)
5
If ?n does not depend on of customers in
system, ?n ?.
If there are s servers, each with same service
rate, then ?n s? for n ? s and ?n n? for
0 ? n lt s. s? customer service capacity per
unit time r ?/s? utilization factor (traffic
intensity)
The systems we study will have r lt 1 because
otherwise the of customers in the system will
grow without bound
We will be interested in the steady-state
behavior of queueing systems (the behavior for
large t). Obtaining analytical results for N(t),
Pn(t), . . . for arbitrary values of t (the
transient behavior) is much more difficult.
6
Notation for Steady-State Analysis
pn probability of having exactly n customers in
the system L expected number of customers in
the system Lq expected queue length
(doesnt include those being served) W
expected time in the system, including service
time. Wq expected waiting time in the queue
(doesnt include service)
7
Littles Law
For any queueing system that has a steady state
and has an average arrival rate of ?, L
?W For example, if the average waiting time is
2 hours and customers arrive at a rate of
3 per hour then, on average, there are 6
customers in the system. Similarly, Lq
?Wq If ?n ? for all n ? 1 then W Wq
1/? (1/? is the mean
service time here)
8
Benefit of Littles Law
These three relationships allow us to calculate
all four quantities L, Lq, W and Wq (once
one of them is known). L ?W requires a
constant service rate.
9
Birth and Death Processes
Applications in a variety of areas, but in
queueing birth refers to the arrival of a
customer while death refers to the departure of
a customer.
Recall, N(t) of customers in the system at
time t.
Assumptions Given that N(t) n, the pdf
governing the remaining time until the next birth
(arrival) is exp(?n) n 0, 1, 2, . . .
Given that N(t) n, the pdf governing the
remaining time until the next death (service
completion) is exp(?n) n 1, 2, . . .
All random variables are assume to be independent.
10
Transition Rate Diagram
Let pn steady-state probability of being in
state n.
11
Balance Equations Section 16.8
Flow into 0 ? ?1p1 ?0p0 ? flow out of
0 Flow into 1 ? ?0p0 ?2p2 (?1 ?1)p1 ?
flow out of 1 Flow into 2 ? ?1p1 ?3p3
(?2 ?2)p2 ? flow out of 2 Flow into n
?n-1pn-1 ?n1pn1 (?n ?n)pn ?
flow out of n
12
l
l
l
l
l
1
p0
,
m
l
0
1
1
1
0

p0

p1


(
P
P
)


p1


p1
p2

m
m
m
1
1
0
1
m
m
m



1
2
2
2
2
1

l
l
l



. . .
n-1

n


0
pn



p0

m
m
m
0


n

n-1

1
l
l
l


n-1

n


0
Let Cn n 1, 2,
and let C0 1

m
m
m


n

n-1

1
Thus pn Cnp0 , k 1, 2, . . . and Snpn
1 so p0(C0 C1 C2 ) 1 or p0 1 /
(1 Sn Cn)


13
Some Steady-State Results
Once we have calculated the pns we can find L
S npn expected of customers in the
system Lq S (ns)pn expected of
customers in queue Ls L Lq expected
of customers in service l S ?npn average
arrival rate E Ls / s efficiency of system
(utilization)
n0
ns1
n0
14
Queueing Example with 3 Servers

• Assume
• Average arrival rate l 5/ hr
• Average service rate m 2 / hr
• Arriving customer balks when 6 are in system.
• 4. Steady-state probabilities

15
Determining System Characteristics
What is the probability that all servers are idle?
Prall servers idle ? 0 0.068
What is the probability that a customer will not
have to wait?
Prnot wait ? 0 ? 1 ? 2 0.45
What is the probability that a customer will have
to wait?
Prwait 1 Prno wait 0.55
What is the probability that a customer balks?
Prcustomer balks ? 6 0.102
16
Steady-State Measures for Example
Expected number in queue Lq 1? 4 2 ? 5 3
? 6 0.700 Expected number in service Ls ?
1 2 ? 2 3(1 ?0 ?1 ?2) 2.244 Expected
number in the system L Lq Ls
2.944 Efficiency of the servers E Ls / s
2.244 / 3 0.748 or 74.8
17
Littles Law with Average Arrival Rate
Applying Littles Law with ? we can calculate
W L / l Wq Lq / l
Expected waiting time in the system
expected waiting time in the queue
Results assumes that steady state will be reached.
18
Example with 3 Servers (cont.)

• To compute average waiting times we must first
  find the average throughput rate
• l S ?npn which simplifies to
• l(1 p6) 5( 1 0.102) 4.488 / hour

n0
19
M/M/1 Queue
M / M / 1 arrival service
of process process servers
M stands for Markovian, meaning exponential
inter-arrival exponential service times.
20
Derivation of Steady-State Probabilities

1
S
n

Recall
x


provided x lt 1


1x
n0
Thus p0 1r and pn r n(1- r).
21
Performance Measures for M/M/1 Queue
l
L --- provided ? ? ?
m - l
l2
l
Lq L - (1 - p0)
-


m
m(m-l)
22
The important thing is not the specific M/M/1
formulas. Methodology used to find these
results is important.
- Model the system as a birth-and-death process
and construct the rate diagram. Depending on
the system, defining the states may the first
challenge. - Develop the balance equations -
Solve the balance equations for pn, n 0,1,2, .
. . - Use the steady-state distribution to derive
L and Lq and, use Littles law to get W and Wq.
23
Example of M/M/1/2 Queue
? A maintenance worker must keep 2 machines in
working order. The two machines operate
simultaneously when both are up. ? The time
until a machine breaks has an exponential
distribution with a mean of 10 hours. ? The
repair time for the broken machine has an
exponential distribution with a mean of 8
hours ? The worker can only repair one
machine at a time.
24
Issues in Evaluating Performance

• Model system as a birth-and-death process.
• Develop the balance equations.
• Calculate the steady-state distribution pn
• Calculate and interpret L, Lq, W Wg.
• What is the proportion of time the repairman is
  busy?
• What is the proportion of time that a given
  machine e.g., machine 1, is working?

25
State-Transition Diagram
? rate at which a single machine breaks down
1/10 hr ? rate at which machines are
repaired 1/8 hr State of the system of
broken machines.
26
Balance Equations for Repair Example
?p1 2?p0 2?p0 ?p2 (?
?)p1 ?p1 ?p2
27
Here, ?0 2? ?1 ? ?1 ? ?2
? ?2 ?
l
l
l
l
2
l
2
2

C1 C2 and C0 1 (by
definition). Thus p0 0.258 ,
p1 p0 0.412
1
0
0



m
m
m
m

m
2
2
1
1
l
2

m
l
2
2
p2
p0 0.330

m
2
L 0p0 1p1 2p2 1.072 (avg machines in
system) Lq 0p1 1p2 0.33 (avg waiting for
repair)
28

?npn ?0p0 ?1p1 ?2p2
(2?)p0 ?p1 0.0928
S
? average arrival rate
ns
1
1
1
1
(0.33)
Wq
Lq
(1.072)
W L
0.0928
0.0928
hours
3.56
11.55 hours
Average amount of time that a machine has to
wait until the repairman initiates the work.
Average amount of time that a machine has to
wait to be repaired, including the time until
the repairman initiates the work.
29
Multi-Channel Queues M/M/s
Additional measures of performance Efficiency E
r Pr Tq 0 ?n0,s-1 pn Pr Tq gt t (1
Pr Tq 0 ) e-sm(1-r)t for t gt 0
30
Telephone Answering System Example

• Situation
• A utility company wants to determine a staffing
  plan for its customer representatives.
• Calls arrive at an average rate of 10 per minute,
  and it takes an average of 1 minute to respond to
  each inquiry.
• Both arrival and service processes are Poisson.

Problem Determine the number of operators that
would provide a satisfactory level of service
to the calling population.
Analysis r l/sm lt 1 or s gt l/m 10.
31
Comparison of Multi-Server Systems
32
Machine Processing with Limited Space for Work in
Process

• Parts arrive at a machine station at the rate of
  1.5 per minute on average.
• The mean time for service is 30 seconds.
• Both processes are assumed to be Poisson.
• When the machine is busy, parts queue up until
  there are 3 waiting. At that point arrivals sent
  for alternative processing.

Goal Analyze the situation under the criteria
that no more than 5 of arriving parts receive
alternative processing and that no more than 10
of the parts that are serviced directly spend
more than 1 minute in the queue.
33
Solution for M/M/1/4 Model
p0 0.328, p1 0.246, p2 0.184, p3 0.138,
p4 0.104 l 1.5/min, m 2/min r l/m
0.75 Balking probability PF p4 0.104 (does
not meet 5 goal) l l(1 PF) 1.344/min L
1.444 W 1.074 min E 1 p0 67.22 PrTq gt
1 0.225 (see text for computations does not
meet 10 goal)
34
Add Second Machine M/M/2/5 Model
New results PF 0.0068, l l(1 p5) 1.49 L
0.85, W 0.57 min E 37.2 PrTq gt 1 0.049
This solution meets our original goals with the
percentage of balking parts now less than 1 and
the probability of a wait time greater than 1
minute less than 5
35
Queuing Networks
Queuing Network
In many applications, an arrival has to pass
through a series of queue arranged in a network
structure.
36
Jackson Network Definition
1. All outside arrivals to each queuing station
in the network must follow a Poisson
process. 2. All service times must be
exponentially distributed. 3. All queues must
have unlimited capacity. 4. When a job leaves one
station, the probability that it will go to
another station is independent of its past
history and is independent of the location of any
other job.
In essence, A Jackson network is a collection of
connected M/M/s queues with known parameters
37
Jacksons Theorem

• Each node is an independent queuing system with
  Poisson input determined by partitioning, merging
  and tandem queuing example

• Each node can be analyzed separately using M/M/1
  or M/M/S model

• Mean delays at each node can be added to
  determine mean system (network) delays

38
Element of a Queuing Network
39
Jackson Networks
Two-stage example. Each station is M/M/s queue.
40
Computation of Input Rate
Let ?i external arrival rate to station i 1,
. . . , m ?ki probability of going from
station k to i in network ?i total input to
station i In steady state there must be flow
balance at each station.
41
Matrix Form of Computations
Property 1 Let ? be the m ? m probability matrix
that describes the routing of units within a
Jackson network, and let ?i denote the mean
arrival rate of units going directly to station i
from outside the system. Then ? ?(I
?)1 where ? (?1,, ?m) and the components of
the vector l give the arrival rates into the
various station that is, ?i is the net rate into
station i. cc
After the net rate into each node is known, the
network can be decomposed and each node treated
as if it is an independent queuing system with
Poisson input.
42
Decomposition of Jackson Network
Property 2 Consider a Jackson network comprising
of m nodes. Let Ni denote a random variable
indicating the number of jobs at node i (the
number in the queue plus the number in service).
Then Pr( N1 n1, Nm nm ) Pr(
N1 n1) ? ? Pr(Nm nm ) and
Pr(Ni ni ) for all ni 0,1,can be calculated
using the equations for independent M/M/S seen
previously
43
Job Shop Example

• Scenario
• Three products
• Six machines A, B, C, D, E, F
• Each product takes different route
• Data

44
Queuing Network for Job Shop
45
Results for Job Shop Example
46
System Performance Measures

• Manufacturing Lead time
• The average time a product spent in the system.
• Summation of time spend in each M/M/s system
• Work-in-process (WIP) inventory
• Determined with Littles law
• WIP (lead time) ? (order rate).
• Questions
• Can we sum L in each M/M/s System to get WIP

47
System Performance for Job Shop

• WIP obtained using Littles law (lead time ) ?
  (order rate).
• Results show a marked difference between the
  products in terms of lead time and WIP since
  product 1 passes through both stations B and D.

48
The Setup of a Computation Center
A high performance computation center is composed
of three stations, a input processor, the central
computer and a graphic simulation center.
All jobs submitted must first pass through an
input process before moving onto the central
process station for error checking. 80 goes
through and 20 are rejected.
All jobs will pass through the central processor.
However, only 40 are routed to graphic
simulation computers where simulation results are
produced.
Jobs arrive randomly at computer center at an
average rate of 10 per minutes. To handle the
load, each station may have several processor
operation in parallel.
49
The Setup of a Computation Center
We know from previous statistics that the time
for the three steps have exponential
distributions with means as follows 10 seconds
for an input processor. 5 seconds for a
central processor. 70 seconds for a graphic
processor. All queues are assumed to have
unlimited capacity. Goal Model it as a Jackson
Queuing Network Find the minimum number of
processor of each time and computer the average
time require for a job to pass through the
system.
50
(No Transcript)
51
Non-Markov Networks
Assume we have a network with K classes of
customers. Each class k?? K has a fixed routing
through the network. Unlimited capacity at each
node. Arrival and service processes not known but
means and standard deviations of interarrival
times and service times are known. each station
in the network can be modeled as.
View each station as an GI/G/1 queue. ? A Jackson
network can be used to approximate this network.
52
Non-Markov Network Example
Let msi mean processing time at station i for i
1, 2, 3 ssi standard deviation of
processing time at station i Data
53
Example (continued)

• Mean time between arrivals is ma 5 minutes so ?
  0.2/min.
• Mean time between departures at station 1, and
  equivalently the mean time between arrivals at
  stations 2, is the same ? ma.
• Similarly, the departures from stations 2 and 3
  all have the same mean, ma.
• Standard deviation of the time between departures
  sd1, sd2 and sd3, will differ, however, because
  of the joint effects of arrival and service
  variability on departure variability.

The approximate relation is cd2 r2cs2 (1
r2) ca2 and sd cd ma. The departure
coefficient of variation is the same as the
arrival coefficient of variation of the next
stage.
54
Results for Non-Markov Network Example
Queues can be analyzed sequentially starting with
station 1 using the formula
At each station W Wq 1/m . Use Littles law
to find L and Lq with l 0.2/min for each
station.
55
Jacksons Theorem - Application in Packet
Switched Networks
Packet Switched Network