Resource Overbooking and Application Profiling in Shared Hosting Platforms

1
Resource Overbooking and Application Profiling in
Shared Hosting Platforms
Bhuvan Urgaonkar Prashant Shenoy Timothy Roscoe
Presented by Sumit Mittal
2
Abstract

• Shared hosting platforms
• Provisioning CPU and network resources
• Feasibility of overbooking resources
• Benefits of controlled overbooking

3
Overview

• Introduction and Motivation
• Derivation of Resource Requirements
• Overbooking Techniques
• Experimental Evaluation
• Conclusion

4
Introduction

• DDedicated hosting platforms
• Shared hosting platforms

• entire cluster runs a single application
• each element dedicated to single application
• large no. of different 3rd party applications
• each application runs on a subset of nodes
• economics of space, power, cooling and cost

5
System Model

• Cluster of heterogeneous nodes
• Each node has processor, memory and
  network interface(s)
• Nodes connected by a high-speed LAN

6
Terminology

• Applications means a complete service running on
  behalf of an application provider
• Distributed components of application known as
  capsules
• Each capsule runs on a separate node

7
Research Contributions

• Automatic Derivation of QoS requirements
• Revenue maximization through overbooking
• Placement and isolation of antagonistic
  applications

8
Overview

• Introduction and Motivation
• Derivation of Resource Requirements
• Overbooking Techniques
• Experimental Evaluation
• Conclusion

9
Derivation of QoS requirements

• Monitoring applications resource usage
• Derive QoS requirements that conform to observed
  behavior
• Overestimations result in resource idling,
  underestimation in application degradation
• Automatic derivation crucial in systems with
  large no. of applications

10
QoS Requirements Definitions

• QoS requirements defined on per capsule basis
• Concerned with CPU, network bandwidth
• Defined in OS-independent manner
• Represented by a quintuple (s, r, t, U, O)

11
Token Bucket Parameters (s, r)

• s is rate of consumption of CPU cycles or
  network interface bandwidth
• r is maximum burst size
• Over interval t, resource usage is s.t r

12
Period t

• Time period for which capsule desires guarantee
  on resource availability
• System to meet requirements over each interval t
• Capsule to be allocated s.t r for each t
• Smaller value of t means more stringent
  requirements

13
Usage Distribution U

• Probability distribution of resource usage
• U(x) denotes probability that capsule uses
  fraction x of resource 0 lt x lt 1
• Necessary for providing probabilistic guarantees
• More detailed specification than (s, r)

14
Overbooking Tolerance O

• Minimum level of service acceptable to the
  capsule
• Probability with which requirements may be
  violated
• e.g. if O 0.01, requirements to be met 99 of
  the time.

15
Kernel-based Profiling

• Run application on set of isolated platform nodes
• Subject application to realistic workload
• Use Linux trace toolkit to monitor CPU and
  network activity

16
Empirical Derivation of QoS Requirements
Begin CPU quantum/Network transmission
End CPU quantum/Network transmission
Time ?
Idle period (OFF)
Busy period (ON)
17
Measurement Interval I
I
I
Time ?
Bucket parameters
s1t r1
Usage Distribution
1
Cumulative resource usage
s2t r2
Probability
0
1
Fraction resource usage
time
18
Derivation of QoS requirements

• Many valid (s, r) pairs for a given usage
• Can use overbooking tolerance O to decide upon
  bucket parameters
• Overbooking tolerance O, period t provided by
  the application

19
Overview

• Introduction and Motivation
• Derivation of Resource Requirements
• Overbooking Techniques
• Experimental Evaluation
• Conclusion

20
Resource Overbooking Techniques

• Resource requirements of existing capsules not to
  be violated
• Sufficient resources to meet requirements of the
  new capsule
• Overbooking tolerances should not be exceeded

21
Test 1 Resource requirements of the new and
existing capsules should be met
k 1
S (si . t min ri) . (1 - Oi) lt C . tmin
i 1
22
Test 2 Overbooking tolerances of all capsules
are met
Pr ( Y gt C) lt min ( O1, O2, , Ok1 )
Y aggregate resource usage C CPU or network
capacity
23
Capsule Placement Algorithms

• Model placement problem using graph with a vertex
  for each capsule and each Node
• If a node feasible for capsule, add an edge from
  capsule to the node
• Bipartite graph connecting capsules to nodes

24
Capsules
Nodes
25
Capsule Placement Algorithm

• Place most constrained capsule on any feasible
  node
• Node and all its edges deleted
• Pick next most constrained capsule, repeat the
  process

Such a greedy algorithm will always find a
placement if it exists !!
26
Choosing a Feasible node

• Three options when more than 1 feasible node
  for a capsule

• Best fit Choose node with least unused
  resources
• Worst fit Choose node with most unused
  resources
• Choose node having capsules with similar
  overbooking tolerances

27
Policy constraints on Capsule Placement

• Consider externally imposed policies which might
  constraint placement
• e.g Allocation of capsules from competing
  applications on different nodes
• Enhance bipartite graph with weights, weights
  measure of external policies
• Attempt to maximize sum of weights of edges
  chosen

28
Overview

• Introduction and Motivation
• Derivation of Resource Requirements
• Overbooking Techniques
• Experimental Evaluation
• Conclusion

29
Experimental Evaluation

• Cluster of Linux-based servers as shared hosting
  platforms
• Each servers runs a QoS-enhanced Linux kernel
• Control plane for shared platform implements
  resource overbooking and capsule placement
  strategies

30
Benefits of resource overbooking in web hosting
platform
31
Observations

• Larger the tolerance of overbooking, larger the
  number of web servers that were run
• For 128 node platform, number of web servers
  increases from 307 to over 1800 for 10 tolerance
• Even for 1 overbooking, factor of 2 increase

32
Resource overbooking for a less bursty streaming
server application
33
Resource overbooking for Application mixes
34
Capsule Placement Algorithms

• Constructed two application types replicated
  web server and e-commerce application
• Each application consists of 2-10 capsules
• Overbooking tolerance set to 5

35
Placing diverse applications
36
Placing similar applications
37
Overbooking conscious placement
38
Overview

• Introduction and Motivation
• Derivation of Resource Requirements
• Overbooking Techniques
• Experimental Evaluation
• Conclusion

39
Conclusions

• Provisioning based on worst case needs results in
  low average utilization
• Controlled overbooking leads to increased
  utilization
• Overbooking by as little as 1 increases
  utilization by a factor of 2
• Benefits more for more bursty applications

40
Questions !!