Performance improvement through Active Idleness

1
Performance improvement through Active Idleness
José A.A. Moreira Agilent Technologies Germany
Carlos F.G. Bispo Instituto de Sistemas e
Robótica Instituto Superior Técnico Portugal
2
Outline

• Motivation
• Active Idleness
• The Time Window Controller
• Implications on Performance
• Conclusions

3
Motivation Queuing network

• Multiple servers
• Multiple types of customers
• Different routs for each type through the network
• Each type of customer may visit same server more
  than once
• A type of customer is composed of several classes
• One per operation needed
• Each server processes different classes with
  different processing time distributions no
  redundant servers
• Each type of customer arrives to the system
  randomly in time but to one unique point and
  exits the system at a specific server
• Customers queue up in front of the server they
  require next

4
Motivation The concept of load

• Assume there are K different classes of customers
• Some may belong to the same type of customer
• Define as ?k the first moment of the arrival rate
  distribution of class k, for k 1, 2, , K, to
  its server
• Define as ?k the first moment of the service time
  distribution of class k, for k 1, 2, , K
• Assume there are I different servers
• Let c(i) designate the constituency of server i,
  for i 1, 2, ..., I
• Class k belongs to c(i) if server i processes it
• Then, the load of server i is
• ?k ?c(i) ?k ?k

5
Motivation Admission and scheduling

• A policy to control the entry of new customers in
  the system is said to be an admission policy
• A policy to decide which customer a server has to
  process next is said to be a scheduling policy
• An open network does not have an admission policy
• Otherwise the system is termed as a closed
  network
• All networks must have a scheduling policy
• If they exist, both policies affect the arrival
  rates of classes to their servers
• In an open network only the internal arrival
  rates are affected
• We address open networks with distributed
  policies (why?)

6
Motivation Stability

• There are many alternative ways of defining
  stability for queuing networks
• For our purposes, suffices to say that a network
  is stable if the expected queue lenght of all
  servers is finite or if the expected time to flow
  through the network is finite for all types of
  customers
• Connected to the above, one can also say that a
  network is stable if all the internal arrival
  rates match the external arrival rates on a short
  term basis
• This is not a precise concept, serves only to
  provide intuition

7
Motivation Traffic Intensity Condition

• The necessary condition for stability states that
  if a network is stable then the load imposed on
  each server by the classes it serves is below
  unity
• Is the TIC also sufficient?
• For many years it was thought so, as long as the
  scheduling policy does not keep servers idle in
  the presence of customers non-idling policies
• Also termed by some authors as work-conserving
  policies
• In the late 80s and early 90s some examples
  were published where, although the TIC holds, the
  networks are unstable when controlled by
  non-idling policies

8
Motivation Dais network

• Parameters
• ?1 1
• ?1 0.001
• ?2 0.897
• ?3 0.001
• ?4 0.001
• ?5 0.001
• ?6 0.899
• Loads
• Server 1 0.9
• Server 2 0.9
• Scheduling
• First Come First Serve

9
Motivation Simulation results
10
Active Idleness What conditions stability?

• The network alone?
• The network plus the scheduling policy?
• What is the network alone?
• Just the servers?
• Servers and routs of customers?
• The above plus the arrival rates and processing
  times?
• What should condition stability?
• Are we interested in determining if a pair
  topology/scheduling policy is stable?
• Or are interested in determining if a given
  topology is stabilizable?

11
Active Idleness Control Theory

• From a control theory perspective
• Is a system stable when left alone?
• If a system is unstable, can we stabilize it
  through the ade quate choice of control?
• Is a network stable when left alone?
• NEVER needs control policy (admission and/or
  scheduling)
• Do we want to know if a network can be stabilized
  through the adequate choice of scheduling policy?
• Does that entail finding the right policy?
• Or does it entail someting else?

12
Active Idleness Wondering

• From the examples published
• It is odd that always working in the presence of
  customers does not ensure stability whenever the
  TIC holds for distributed policies
• There are starvation periods being created, which
  translate into short term loss of capacity, that
  will not be recovered
• Should we blame it on the policy alone?
• Or is there something else?
• I wonder...
• What if we drop the requirement of always using
  non-idling policies?
• What if we allow servers to stop in the presence
  of customers?
• Is there a long term capacity gain due to
  controlled short term losses of capacity?

13
Active Idleness Wondering still

• Surely you are kidding...
• The network is unstable and you want to stabilize
  it by wasting capacity!!!...
• Find a job somewhere else...
• Well...
• We are wasting capacity for the fact that we do
  not filter the burstiness of the arrival
  processes
• Some servers get lots of customers of a given
  class, work on them and the next server will get
  a handfull of new customers, but some other
  server is not getting anything, thus the waste

14
Active Idleness Key to stability?

• Active Idleness is perhaps one key
• Dont be passive about staying idle
• Chose your moments of idleness and perhaps you
  will not have to be idle for such long periods
• Filter burstiness
• Block classes that are flooding the server,
  allowing for some others to get through
• If a class gets blocked and there are others,
  work on them
• If there are no other classes, what is the rush
  of working on a class that you have been working
  for a significant amount of time?

15
The Time Window Controller

• It is one possible implementation of the AI
  concept
• Time window of size T (finite)
• Processing History - Look T units into the past
  and compute the fraction of time each class used
  its server
• Maximum time fraction assign a maximum value to
  each class
• Blocking occurs when a class has exceeded its
  maximum time fraction
• Server only sees classes which are not blocked,
  the blocked ones are assumed not be present while
  they remain blocked
• Not a new scheduling policy
• decisions are still made by the same criteria as
  before, but only on the visible classes
• As time progresses, classes leave the blocking
  status and become elegible to be processed
  according to the policy

16
Implications on Performance

• Simulation results - stability

17
Implications on Performance

• So we manage to stabilize unstable networks
• What if the pair network/scheduling policy is
  already stable?
• Is there any advantage on using the Time Window
  Controller?
• Take the same network as before and use Last
  Buffer First Serve
• We know this pair to be stable when the TIC holds
• Use some metric to evaluate performance
• Higher cost to earlier stages
• Average queue lenght

18
Implications on Performance

• Simulation results improvement over stable
• Higher cost to earlier stages

19
Implications on Performance

• Active Idleness as a function of fraction
• Higher cost to earlier stages

20
Implications on Performance

• Simulation results improvement over stable
• Average bufer lenght

21
Conclusions What was done

• Proposed to use Active Idleness as a way to
  ensure stabilizability
• Described a controller implementing the concept
• Shown the dramatic improvement for an unstable
  system
• Shown the potential to optimize performance,
  irrespective of the fact that original system is
  or is not stable

22
Conclusions What remains to do

• Would like/need to develop an efficient way to
  tune the controller
• Brute force parameter tunning
• Infinitesimal Perturbation Analysis
• Other...
• What if buffers are finite?
• What if there is a set-up time associated with
  changing classes?
• How about spliting and merging?
• What is the best distributed scheduling policy
  with AI?
• In general, is it the case that the best policy
  is non-idling?

23
Performance Improvement through Active Idleness
José A.A. Moreira jose_moreira_at_agilent.com
Carlos F.G. Bispo cfb_at_isr.ist.utl.pt http//www.is
r.ist.utl.pt/cfb