ClassDependent Assignment in Clusterbased Servers

1
Class-Dependent Assignment in Cluster-based
Servers

• Victoria Ungureanu
• Benjamin Melamed
• Phil Bradford
• Michael Katehakis

2
Background

• Cluster-based Servers

Dispatcher
Back-end Server 1
Back-end Server 2
Back-end Server n
3
Background (cont.)

• Internet Traffic
• Follows a power-law distribution
• A relatively small fraction of jobs accounts for
  a relatively large fraction of the overall load
• For example, the data traces from sites serving
  the 1998 World-Cup
• Files with sizes lt 2KB, make up 75 of the
  files requested, and account for 12 of the
  transferred data
• Files with sizes gt30KB, make up 3 of the files
  requested, and account for 50 of the transferred
  data
• Files with sizes gt100KB, make up 0.04 of the
  files requested, and account for 7 of the
  transferred data

4
The Problem

• Devise an assignment policy for cluster-based
  servers that
• Achieves good response time performance
• Join Shortest Queue(JSQ) a job is assigned to
  the back-end server with the least amount of
  residual work
• Size-Range a back-end server is assigned only
  jobs of similar sizes
• Is Practical to implement
• Round-Robin
• Least Connected a job is assigned to the
  back-end server having the least number of jobs

5
Class Dependent Assignment (CDA)

• The dispatcher has a cutoff parameter c for
  classifying jobs into long and short
• Short jobs are assigned in Round-Robin manner, as
  soon as thy arrive
• Long jobs are not assigned as they arrive, but
  are hold in a dispatcher queue
• When a back-end server becomes idle, it is
  assigned a long job
• While processing a long job, a back-end server
  is not assigned any other jobs.

6
The Rationale for CDA

• Achieves good performance because
• It reduces the variance of job-sizes assigned to
  a back-end server ? short jobs are not stuck
  behind long ones
• It gives priority to the many short jobs, by
  deferring long jobs
• It is practical to implement because the
  dispatcher needs only
• to estimate the size of long jobs, and
• to know when a back-end server becomes idle.
• ? low overhead incurred by assignment.

7
On the Implementation of CDA

• The classification into long and short jobs is
  based on the size of requested documents
• Dispatcher has this type of information readily
  available.
• The dispatcher periodically updates estimated
  sizes of large jobs in its queue
• the estimated size decreases with the time a
  job has waited
• Prevents the starvation of large jobs.

8
Simulation Experiments

• Compares the performance of CDA with Round-Robin
  and JSQ
• Performance metric waiting time (the time
  interval from the moment a request arrives at the
  dispatcher and up until it starts processing at a
  back-end server)
• The study models a cluster of 4 back-end servers,
  and makes the assumptions that
• Communication times between the dispatcher and
  back-end server is negligible
• The time to select (job,back-end server) is
  negligible
• There is no job preemption---a back-end server
  finishes a job, before starting another.

9
Experimental Performance Study

• Uses a fragment from World-Cup traces, covering
  40 minutes, and containing 3.5 million requests

Legend x-axis time (s) y-axis number of
requests
10
File-size Distribution of Requests for World-Cup
Traces

Legend x-axis file size (KB) y-axis number of
requests (logarithmic scale)
11
The Effect of the Cutoff Point Value on CDA
Performance
Legend x-axis cutoff point (KB) y-axis average
waiting time (micros)

12
Performance Comparison of CDA and JSQ for a
Fixed Value of Cutoff

Legend x-axis time (time unit 10
s) y-axis average waiting
time cutoff 30KB
13
Local vs. Global Information

• CDA
• The dispatcher gives priority to short jobs
• The back-end servers process the jobs in the
  chronological order of their arrival.
• ? it uses global information available to the
  dispatcher
• SJF (Shortest Job First)
• The dispatcher assigns all requests in
  Round-Robin manner
• A back-end server selects for processing the
  shortest job currently assigned to it.
• ? SJF uses local information, available to
  back-end servers

14
Local vs. Global Information (cont.)
Legend x-axis time (min) y-axis average
waiting time (micros)
CDA SJF

15
Limitations of CDA

• The dispatcher needs to know the size of a
  requested document
• Dynamic requests are automatically classified as
  short
• The method is applicable to sites that serve
  preponderantly static requests
• The study assumes that there is no job
  preemption
• the results apply to multi-threaded back-end
  servers using co-routine scheduling
• the performance of CDA may change if back-end
  servers process jobs concurrently.

16
Conclusion

• CDA
• Achieves good performance the average waiting
  time of CDA, for c30K is
• two orders of magnitude better than Round-Robin
• 40 better than SJF
• similar with JSQ
• It is practical to implement modest
  informational requirements
• Prevents starvation of large jobs