SCHEDULING FOR TODAY

1
SCHEDULING FOR TODAYS COMPUTER SYSTEMSBRIDGING
THEORY AND PRACTICE
Adam Wierman
2
SCHEDULING SUCCESS STORIES ARE EVERYWHERE
Biersack, Rai, Urvoy-Keller, Harchol-Balter,
Schroeder, Agrawal, Ganger, Petrou, Misra, Feng,
Hu, Zhang, Mangharam, Sadowsky, Rawat, Dinda,
McWherter, Ailamaki, others
3
THE ESSENCE OF A SCHEDULING SUCCESS STORY
web server, edge router, etc.
Goal Minimize user response times
4
SRPT WINS BIG
WHAT POLICY SHOULD WE USE?
SRPT
Assumption M/GI/1 Queue
5
HOW DO REAL SYSTEMS DIFFER?
6
3 TYPES OF GAPS BETWEEN THEORY AND PRACTICE
Idealized policiesThe idealized policies studied
in theory cannot be used in practice
Limited metricsMany performance metrics that are
important in practice are not studied in theory
Simplistic modelsTraditional models include many
unrealistic assumptions
7
THE GOAL OF THE THESIS BRIDGE THE GAPS BETWEEN
THEORY AND PRACTICE
8
IDEALIZED POLICIES
SRPT
9
SRPT
10
IDEALIZED POLICIES
SRPT
SMART
SMARTe
11
THE SMART CLASS
SMAll Response Times

1. Bias Property
2. Consistency Property
3. Transitivity Property

12
TWO NOTIONS OF SMALL JOBS
small original size
small remaining size
13
BIAS PROPERTY
If OriginalSize(a) lt RemainingSize(b) then a has
priority over b
a
b
Sigmetrics 2005a
14
BIAS PROPERTY
If OriginalSize(a) lt RemainingSize(b) then a has
priority over b
Sigmetrics 2005a
15
BIAS PROPERTY
If OriginalSize(a) lt RemainingSize(b) then a has
priority over b

lower priority
remaining size

?
higher priority

0
original size
0
Sigmetrics 2005a
16
EXAMPLES
If OrigSize(a) lt RemSize(b) then a has priority
over b
Many others!
17
IDEALIZED POLICIES
SRPT
SMART
SMARTe
18
ANALYSIS SETTING
APPROACH
ETSMART
Bound T(x)SMART
19
CONDITIONAL RESPONSE TIME UNDER SMART POLICIES
Theorem Under the M/GI/1, for all SMART policies
P,
Response time for a job of size x
Sigmetrics 2005a
20
CONDITIONAL RESPONSE TIME UNDER SMART POLICIES
Theorem Under the M/GI/1, for all SMART policies
P,
Picture proof Waiting time
remaining size
?
21
SMART POLICIES ARE 2-COMPETITIVE
Sigmetrics 2005a
22
SMART POLICIES ARE 2-COMPETITIVE
SRPT
23
IDEALIZED POLICIES
SRPT
SMART
SMARTe
24
SMART
SMARTe
If OrigSize(a) x and e(x) lt RemSize(b) then a
has priority over b
If OrigSize(a) lt RemSize(b) then a has priority
over b
25
SMARTe
If OrigSize(a) x and e(x) lt RemSize(b) then a
has priority over b
How can you characterize job size estimates?
e(x) x error
e(x) can also be defined to include 2-level
policies
26
SMARTe POLICIES ARE CONSTANT COMPETITIVE
Theorem In an M/GI/1 under SMARTe policy P
f bounds the SIZE of larger jobs that get higher
priority
d bounds the LOAD of larger jobs that get higher
priority
27
real sizes ? web server trace
estimates ? within 50
WHAT DOES THIS TRANSLATE TO IN PRACTICE?
SMARTe
SRPT
28
IDEALIZED POLICIES
SRPT
SMART
SMARTe
29
MUCH MORE WORK ON CLASSIFICATIONS
Analyzing the SMART class beyond ET
Introducing analyzing other classifications
Sigmetrics 2005a Sigmetrics 2006 Perf. Eval.
Review 2006 Operations Research 2007

Sigmetrics 2003Sigmetrics 2005b Perf. Eval.
Review 2006
Collaborations with Zwart, Nuyens, Shakkottai,
Yang, Harchol-Balter, Osogami, and others
Freidman and Hurley, 2004Rai, Urvoy-Keller,
Vernon, Biersack 2005 Nunez-Queija, Kherani
2006 Misra, Rubenstein, Feng 2007 Kherani
2007
30
THE GOAL OF THE THESIS BRIDGE THE GAPS BETWEEN
THEORY AND PRACTICE
10 of the talk
30 of the talk
60 of the talk
31
MEAN RESPONSE TIME
32
MEAN RESPONSE TIME
Perf Eval 2002 Sigmetrics 2003 Sigmetrics
2005a PER 2007
Sigmetrics 2005bSigmetrics 2006 OR 2007
33
ARE POLICIES THAT PRIORITIZE SMALL JOBS UNFAIR
TO LARGE JOBS?
34
WHAT DOES FAIRNESS MEAN?
...it depends entirely on the application
OUR SETTING Are the response times of large jobs
unfairly long?
35
WHY IS THIS FAIR?
Aristotles notion of fairness Like cases should
be treated alike, different cases should be
treated differently, and different cases should
be treated differently in proportion to their
differences.
Sigmetrics 2003 Best student paper award
36
WHY IS THIS FAIR?
Rawls Theory of Social Justice All social
goods should be distributed equally,
unless unequal distribution is to the advantage
of the least favored
Sigmetrics 2003 Best student paper award
37
WHY IS THIS FAIR?
Min-Max fairness (Pareto Efficiency) All jobs
deserve an equal share of the resources ... but
if some jobs can use more without hurting others,
thats okay
Sigmetrics 2003 Best student paper award
38
HOW UNFAIR ARE SMART POLICIES?
Theorem For all service distributions, SRPT is
fair if ?0.5.
Theorem For all power law (a) service
distributions with a lt 1.5, all SMART policies
are fair.
39
BUT SMART POLICIES CAN BE UNFAIR
SMART
the largest jobs are treated the same as under PS
Theorem For all service distributions with
finite variance, all SMART policies are unfair
for high enough load.
40
FAIRNESS AND CLASSIFICATIONS
Always Unfair
Always Fair
SRPT
PS
Sometimes Fair
Sigmetrics 2003 Best student paper award
41
FAIRNESS AND CLASSIFICATIONS
Always Unfair
Always Fair
Remaining size based
LRPT
SRPT
SMART
FOOLISH
PLCFS
Preemptivesize based
PSJF
PLJF
SYMMETRIC
LAS
Age based
PS
Sometimes Fair
FCFS
PROTECTIVE
Non-preemptive non-size based
LCFS
FSP
Sigmetrics 2003 Best student paper award
42
MUCH MORE WORK ON FAIRNESS
defining other types of fairness
analyzing policies classifications
Under submission
Perf. Eval 2002 Sigmetrics 2003
Raz, Avi-Itzhak 2004 Levy, Raz, Avi-Itzhak
04 Sandmann 2005
extending definition to higher moments
Many other papers by Henderson,
Friedman, Biersack, Rai, Ayesta, Aalto,
Nunez-Queija, Misra, Feng, Vernon, Williamson,
Brown, Bansal, and others
Sigmetrics 2005b Under submission
Williamson, Gong 2003, 2004 Brown 2006 and
others.
43
THE GOAL OF THE THESIS BRIDGE THE GAPS BETWEEN
THEORY AND PRACTICE
10 of the talk
30 of the talk
60 of the talk
44
M/GI/1
45
M/GI/1
PER 2004QUESTA 2005 Perf Eval 2006
NSDI 2006
46
REAL USERS ARE INTERACTIVE
47
Closed System
users75
NSDI 2006
48
REAL USERS ARE NOT OPEN OR CLOSED
load 0.7
CLOSED
OPEN
NSDI 2006
49
Kasparov vs. Deep Blue
world cup site
online shopping
CMU web server
online gaming site
slashdotted site
CLOSED
OPEN
NSDI 2006
50
USER BEHAVIOR IMPACTS SYSTEM DESIGN
51
THE GOAL OF THE THESIS BRIDGE THE GAPS BETWEEN
THEORY AND PRACTICE
10 of the talk
30 of the talk
60 of the talk
52
THE GOAL OF THE THESIS BRIDGE THE GAPS BETWEEN
THEORY AND PRACTICE
Fairness QoS Pr(Tgtx)
Interactive users Multiserver systems
Scheduling classifications
53
Adam WiermanCarnegie Mellon Universityacw_at_cs.cmu
.eduThe thesis is available at
http//www.cs.cmu.edu/acw/thesis
54
(No Transcript)
55
SMART?
FOOLISH
SMART
FOOLISH
SMART
Remaining size based
LRPT
SRPT
RS
Preemptive size based
PSJF
PLJF
FSP
PROTECTIVE
Non-preemptive
Blind
PS
PLCFS
Non-preemptive size based
ROS
SYMMETRIC
LCFS
SJF
FCFS
Age based
LJF
FB
56
THE BIAS PROPERTYISNT ENOUGH
CONSISTENCY
If a is served ahead of b then a will always have
priority over b


TRANSITIVITY
If an arriving job b preempts c, then until b
leaves, every arriving job a with original size
smaller than b has priority over c.
remaining size
?

at most 1 has higher priority
orig. size
Sigmetrics 2005a
57
Theorem
2b
58
DO YOU USE AN OPEN OR CLOSED MODEL?
Surge
SPECWeb
TPC-W
Sclient
RUBiS
WebBench
Webjamma
httperf
59
Theorem In an M/GI/1 with an unbounded, continuou
s service distribution having finite EX2, under
any non-idling policy we have and further
Wierman and Harchol-Balter 2003
60
First Come First Served
The unfairness can be unbounded
ET(x) / x
PS
FCFS
x
Under a Pareto with ?0.8, this is gt80 of the
jobs
61
FAIRNESS VS. EFFICIENCY
Always Unfair
SRPT
LRPT
Always Fair
PLJF
PSJF
LCFS
Sometimes Fair
ROS
FCFS
SJF
LJF
more circles? better mean response time
FB
62
Fair Sojourn Protocol (FSP)
Do SRPT on the PS remaining times
63
Raw moments ET(x)i Central moments
VarT(x), etc Cumulant moments
ET(x) ? ?
X
BEYOND EXPECTATION Higher Moments
X
Wierman and Harchol-Balter 2005
64
CUMULANTS
Cumulants are a descriptive statistic, similar to
the moments. They can be found as a function
of the moments or from the log of the moment
generating function
65
WHYCUMULANTS?
Cumulants have many nice properties
additivity
homogeneity
1st cumulant is shift-equivariant the rest are
shift-invariant
66
MIN-MAX FAIRNESS
Wierman and Harchol-Balter 2005
Definition Consider an M/GI/1 queue. A policy P
is min-max fair if, for all i
67
TEMPORAL FAIRNESS
It is unfair to violate the seniority of a job
Definition The politeness experienced by a job
of size x under policy P, Pol(x)P, is the
fraction of the response time during which the
seniority of the job is respected.
Wierman 2004
68
FCFS
FCFS
SRPT
FSP
PS
PLCFS
FSP
SRPT
PS
LRPT
LRPT
PLCFS
69
FCFS
SRPT
FSP
PS
more circles? better mean response time
LRPT
PLCFS
70
MANY OTHER INTERESTING FAIRNESS METRICS
Levy, Raz, Avi-Itzhak 04

• DiscFreq ni cmi
• ni number of jobs that arrived later and
  completed earlier than job i
• mi number of larger jobs (at the arrival of job
  i) that complete earlier than job i

71
SMART POLICIES ARE 2-COMPETITIVE
Consider the M/D/1
SRPT does FCFS (only in M/D/1). So as ??1
PLCFS is in SMART (only in M/D/1)
As ??1, ETPLCFS ? 2 ETSRPT
72

1. Use a parameterized policy setthat is (nearly)
   dense in SMART,e.g. iRj S
2. Search (i,j) space for policy thatoptimizes
   secondary objectives,e.g. fairness and
   predictability

ONLINE MULTI-OBJECTIVE SCHEDULING USING SMART
73
B
A
B
C
A
D
remaining size
PSJF
D
C
time
74
B
A
?
B
A
D
remaining size
PSJF
PSJF
D
C
SRPT
time
75
TAIL BEHAVIOR OF SMART
Pr(Tgty) is difficult to study directly so it is
typically it is studied asymptotically
76
LARGE BUFFER SCALING
LIGHT-TAILED JOB SIZES
HEAVY-TAILED JOB SIZES
X is light-tailed if for some sgt0 For this
talk, assume no mass in the upper bound.
X is of intermediate regular variation if
77
SMART POLICIES ARE ASYMPTOTICALLY EQUIVALENT

• Theorem Under the GI/GI/1, for all SMART
  policies
• when the service distribution is light-tailed
  with no mass in
• the endpoint
• when the service distribution is of intermediate
  regular variation

busy period length
Nuyens, Wierman, Zwart 2005
78
SMART POLICIES ARE ASYMPTOTICALLY EQUIVALENT
LIGHT-TAILED JOB SIZES
HEAVY-TAILED JOB SIZES
worse
worse
SMART LCFS SJF
Pr(Tgty) workload
Log Pr(Tgty) busy period
SMART
Pr(Tgty) busy period
Log Pr(Tgty) workload
Nunez-Queija, Boxma, Zwart, Borst, Nuyens, and
many others
79
TAIL BEHAVIOR OF SMART
Pr(Tgty) is difficult to study directly so it is
typically it is studied asymptotically
80
MANY SOURCES SCALING
decay rate
Yang, Wierman, Shakkottai, Harchol-Balter, 2006
81
T(x) RESULT, plot for ET(x)
SMART POLICIES ARE ASYMPTOTICALLY EQUIVALENT
Theorem For all e, x, y gt 0 where PRIO is a 2
class priority queueing policy.
Empty!
Picture proof
Yang, Wierman, Shakkottai, Harchol-Balter, 2006
82
T(x) RESULT, plot for ET(x)
TAIL BEHAVIOR OF SMARTe POLICIES

• Theorem Under the M/GI/1, for all SMARTe
• when the service distribution is unbounded and
  light-tailed
• with no mass in the endpoint
• when the service distribution is of intermediate
  regular variation
• and

busy period length
Nuyens, Wierman, under preparation
83
HOW QUICKLY DOES CLOSED ? OPEN?
Open
Web Workloads
Closed (MPL1000)
Closed (MPL100)
Closed (MPL10)
84
CHOOSING A SYSTEM MODEL
Web workloads

1. A site being Slashdotted
2. Online gaming site
3. Science Institute - USGS
4. Online dept. store
5. Financial service provider
6. Kasparov vs Deep Blue
7. CMU web server
8. World cup site

Open or closed?
Use a partly-open model...
85
FITTING A PARTLY-OPEN MODEL
file sizes from trace
86
FITTING A PARTLY-OPEN MODEL

• Fitting the interarrival times
• Distinguish userse.g. use ip address in a web
  trace
• Identify user session boundaries ? Use periods
  of inactivity of length gt timeout

Cant use trace directly because
dependencies between completions and follow-up
requests would be lost!
87
CHOOSING A TIMEOUT VALUE
88
HOW TO CHOOSE A SYSTEM MODEL
How many simult. users are there?
Gather a trace
89
MULTISERVER QUEUES

• Preemptive-Resume Priority
• Homogeneous hosts

90
HOW MANY SERVERS ARE BEST?
1 fast (rate 1) vs. k slow (rate 1/k)
4 best
3 best
2 best
1 best
91
GAPS BETWEEN THEORY AND PRACTICE
What about power usage?
What about time-varying workloads?
What about user impatience?
92
warning
Remember