Influence of heavy-tailed distributions on load balancing

1
Dimensionality Reduction for the analysis
of Cycle Stealing, Task Assignment, Priority
Queueing, and Threshold Policies (PART 1)
Mor Harchol-Balter
Joint with Alan Scheller-Wolf, Taka Osogami, and
Mark Squillante.
2
Many multiserver scheduling problems
FIFO
FIFO
Goal Mean response time per job type
3
Many multiserver scheduling problems
FIFO
FIFO
Common Problem 2D-infinite Markov chain
(or nD-infinite)
4
Dimensionality Reduction (2D) Recursive
Dimensionality Reduction(nD)

• Numerical technique
• (not closed form)
• Problem-dependent.
• Doesnt always apply.

• Provides accurate performance
• numbers for many common problems.
• Very fast (less than 1 sec).
• lt 1 error typically, and can improve.
• Can analyze any load (non-limiting).
• Allows PH service time distributions.
• Adding thresholds and switching costs
• is often easy.

5
Cycle Stealing Problem
lB jobs/sec
lD jobs/sec
Load rB
Load rD
6
Cycle Stealing Problem
lB jobs/sec
lD jobs/sec
Load rB
Load rD
7
Cycle Stealing Problem
lB jobs/sec
lD jobs/sec
Load rB
Load rD
switch back
When new donor job arrives, donor switches back
to donor queue.
8
Generalized Cycle Stealing Sigmetrics 03
lD jobs/sec
Load rB
Load rD
9
Cycle Stealing Problem
lB jobs/sec
lD jobs/sec
Load rB
Load rD
What is Bettys/Dans mean response time?
10
Whats so hard?
Even simplest-case chain grows infinitely in 2D.
11
Prior work cycle stealing coupled-processor
Exponential job sizes (80s)
General Job sizes (80,90,00s)
Truncate the chain
Tail Asymp or Heavy traff.
Bob Foley McDonald Mike Harrison Boxma Borst van
Uitert Williams
Convert to Riemann-Hilbert problem
Convert to Wiener-Hopf boundary problem
Very complex integrals. No numbers
Very complex integrals for WORKLOAD. No numbers.
Fayolle, Iasnogoradski, Konheim,
Meilijson, Melkman ...
Cohen, Boxma, Borst, Uitert, Jelenkovic ...
12
Dimensionality Reduction Key idea
2D-infinite chain
1D-infinite chain
VERY HARD
EASY
13
Whats so hard?
Even simplest-case chain grows infinitely in 2D.
14
Solution

• New type of transition -- Busy period
  transition BD
• 1D-infinite chain

15
Solution
Approximation. But can be made as close to
exact as desired. Tools03a,Tools03b
g

BD
b
a
16
Markov chain is easy to generalize

• Generalize to PH service distribution
• Generalize to switching times
• Generalize to include thresholds

17
MODIFIED 1D-chain cycle stealing with switching
costs and NBth 3
18
Some interesting questions
Q When does cycle stealing pay?
Q How does donor job size variability affect
benef. resp. time?
19
Some interesting questions
Q How should we set NBth?
Q How should we set NDth?
20
Task Assignment Problem
What is a good task assignment policy for
minimizing mean response time?
21
Task Assignment Problem
SHORTS HERE!
High variability job sizes
LONGS HERE!
Supercomputing Applications Non-preemptive
service
Want to isolate short jobs JPDC 99, JACM 02
22
Task Assignment Problem
SHORTS
High variability job sizes
LONGS. But short if idle.
Supercomputing Applications Non-preemptive
service
Even better!
23
Smart Task Assignment
1D-infinite chain
2D-infinite chain
DR
VERY HARD
EASY
24
Results
ETS
Immed. Disp. with sharing
No sharing
Central Queue with sharing
ICDCS 03 SPAA 03
rS
25
So far ...

• Examples of some problems
• - cycle stealing with switching cost
• - task assignment
• where Dimensionality Reduction (DR) is very
  useful.
• Next More problems
• a) Affinity model and more complex threshold
  policies.
• b) Priority queueing in multiserver system and
  
• Recursive Dimensionality Reduction (RDR).
• Finally Current limitations of DR and RDR.