Controlled MMK Queueing Systems

1
Controlled M/M/K Queueing Systems
Efrosinin Dmitry
Notations Markov decision process Optimality
Equation Threshold structure of optimal control
policy Algorithm Examples
2005
2
M/M/K/B queueing system with heterogeneous servers
1
Arriving jobs
Departing jobs



Queue
K
Servers
The usage cost
The holding cost
Arrangement of the servers
Stability condition (B?)
The control times
All arrival and service completion epochs
3
General conditions 1. At an arrival epoch the
control consists in sending the arrived job to
the queue (if it is not full) or to one of
free servers 2. At a service completion epoch,
when the queue is not empty, the control consist
in assigning a waiting job
to one of idle servers, or leave the queue as it
is 3. A job arriving to a full buffer is
rejected in case of full buffer when all the
servers are busy 4. Being
sent to some server a job can not change it
Notation
System state at time t
The queue length at time t
System state
State space
Set of available controls
4
?
Set of admissible controls in case of a new
arrival to state x.
?
Set of admissible controls in case of a service
completion on the j-th server in state x.
the set of labels assigned to idle servers in x.
the set of labels assigned to busy servers in x.
Consider a stationary markov strategy
that always prescribes a single action
in case of a new arrival to state x, i.e.
in case of a service completion on the j-th
server in state x, i.e.
Denote by
the K1 dimensional vector i-th coordinate of
which (beginning from 0-th) is one and all
others are zeros
5
Under the given strategy ? we have
The aim Minimization of the average processing
cost
The total processing cost in the system during
time t
The problem is to find the value of the average
processing cost per unit of time
where
for ergodic markov process
6
Optimality equation
To obtain the optimality equation consider the
function
the minimal total average processing cost over
time t given an initial state x.
Consider the behaviour of this function in shot
time interval ?t
7
If we get the differential
equation
The function V(x,t) has the form
and for large t
Howard (1964)
where is called a
value function and is unknown value.
Now we get the following equation
8
(No Transcript)
9
Theorem 1. The optimal strategy
is determined through the optimal policy which
has the form
Main results (if the servers are arranged in
order
)
Theorem 2. The value function of the model
satisfies the inequality
Corollary 1. The optimal control policy consists
in activating the available fastest server, if
necessary.
Theorem 3. The value function of the model
satisfies the relation
10
Corollary 2. The optimal control policy for the
queue M/M/K is of a threshold type, i.e. for each
state x there exist some level of the queue
length (depending on the collection of
busy servers J1(x) ) such that it is necessary to
switch on the fastest server j among the idle
only if .If in some
state x the optimal decision is to allocate a
customer to the queue, the this decision is
optimal for all y with q(y)ltq(x) and the same
collection of busy servers J1(y) J1(x).
Theorem 4. In case of light traffic
threshold level for the j idle server
is independent of the state of slower servers and
can be calculated analytically
Remark 1. In case of servers arrangement
the previous results do not hold true.
11
Theorem 5. For the queue M/M/K using the
asymptote to the boundary between the region
where the optimal threshold is and
the region with optimal threshold
the following approximations to the optimal
threshold levels can be calculated by for the
threshold levels,
12
Algorithm.
Step 1. Calculation of the value function
For a given strategy
(starting with n0) solve
iteratively a system of linear equations with
respect to and
with given accuracy ?gt0 and initial
values of the first iteration for m0
Step 2. Policy improvement
For a given value function
find a new strategy
which minimizes the right side of the equations
The Algorithm stops when two successive solutions
for policies coincide.
13
One dimensional representation of the system
states
Convergence Satz!!!!!!!
14
Numerical results. Control tables
Iteration procedure.
Threshold formula.
15
Iteration procedure.
Threshold formula.
16
Iteration procedure.
Threshold formula.
17
Iteration procedure.
Threshold formula.
18
Queue length q(x) M/M/5
19
Numerical results. Control diagrams
M/M/3
20
M/M/3
21
M/M/3
22
M/M/3
23
M/M/3
24
M/M/3