Cataclysm: Policing Extreme Overloads in Internet Applications

1
Cataclysm Policing Extreme Overloads in Internet
Applications

• Bhuvan Urgaonkar and Prashant Shenoy
• University of Massachusetts

2
Motivation

• Internet applications used in a variety of
  domains
• Online banking, online brokerage, online music
  store, e-commerce
• Internet usage continues to grow rapidly
• Broadband deployment is accelerating
• Outages of Internet applications more common

Site not responding connection timed out
3
Internet Application Outages
Holiday Shopping Season 2000
Down for 30 minutes
Periodic outages over 4 days
Average download time 260 sec
9/11 site inaccessible for brief periods
Cause Too many users leading to overload
4
Internet Data Centers

• Internet applications run on data centers
• Server farms
• Provide computational and storage resources
• Applications share data center resources

• Problem How can the platform handle extreme
  overloads seen by applications?

5
Handling Extreme Overloads

• Existing work is based on three approaches
• Request policing Kanodia00, Li00, Verma03,
  Welsh03,
• Dynamic capacity provisioning Chase01, Ranjan04
• Degrade performance of admitted requests
  Abdelzaher99
• Shortcomings of existing work
• Does not attempt to integrate these three
  approaches
• Does not address scalability of the policer!
• The policer itself may become the bottleneck
  during overloads

6
Our Contribution Cataclysm

• Comprehensive approach
• Novel policer that can scale during overloads
• Dynamic provisioning for both application and
  policer
• SLA-based performance adaptation
• Implementation and evaluation on a Linux cluster
• Focus of this talk design of the policer

7
Talk Outline

• Motivation
• Internet data center model
• Request policing
• Cataclysm Server Platform
• Experimental results
• Summary

8
Data Center Model
Retail Web site
streaming

• Dedicated hosting each application runs on a
  subset of servers in the data center
• Subsets are mutually exclusive no server sharing
  
• Data center hosts multiple applications
• Free server pool unused servers

9
Internet Application Model
load balancing sentry
requests
dropped requests
http

• Internet applications replicated on multiple
  servers
• E.g., clustered HTTP
• Each application employs a sentry
• Load balancing and request policing
• One or more request classes
• Service-level agreement
• Specifies certain guaranteed request admission
  rate per class
• Specifies allowed degradation in response time
  with arrival rate

10
Talk Outline

• Motivation
• Internet data center model
• Request policing
• Cataclysm Server Platform
• Experimental results
• Summary

11
Policer Design Goals

• Class-based differentiation
• Each class should sustain its guaranteed
  admission rate
• Revenue maximization
• Challenging due to online nature of the problem
• An admitted request may cause a more important
  request arriving later to be dropped
• Approach Preferential admission to higher class
  requests
• Scalability
• The policer should remain operational even under
  extremely high arrival rates

12
Overview of Policer Design
Admission control
dgold
Class gold
admitted
dsilver
Class silver
Classifier
dropped
dbronze
Class bronze
Leaky buckets
Class-specific queues

• Cataclysm policer has three components
• Request classifier and per-class leaky buckets
• Class-specific queues
• Admission control

13
Class-based Differentiation
Admission control
dgold
Class gold
admitted
dsilver
Class silver
Classifier
dropped
dbronze
Class bronze
Leaky buckets
Class-specific queues

• Each incoming request undergoes classification
• Per-class leaky buckets used to ensure that rates
  guaranteed in SLA are admitted

14
Revenue Maximization
Admission control
dgold
Class gold
admitted
dsilver
Class silver
Classifier
dropped
dbronze
Class bronze
Leaky buckets
Class-specific queues

• Idea Add different delays in processing of
  requests of different classes
• More important requests processed more frequently
• Methodology to compute delay values in online
  manner
• Bounds probability of a request denying admission
  to a more important request

15
Admission Control
Admission control
dgold
Class gold
admitted
dsilver
Class silver
Classifier
dropped
dbronze
Class bronze
Leaky buckets
Class-specific queues

• Goal Ensure that an admitted request meets its
  response time target
• Measurement-based admission control algorithm
• Use information about current load on servers and
  estimated size of new request to make decision

16
Scalability of Admission Control

• Idea 1 Reduce the per-request admission control
  cost
• Admission control on every request may be
  expensive
• Bursty arrivals during overloads gt batches get
  formed
• Delays for class-based differentiation gt batches
  get formed
• Admission control test that operates on batches
  instead of requests
• Idea 2 Sacrifice accuracy for computational
  overhead
• When batch-based processing becomes prohibitive
• Threshold-based scheme
• E.g., Admit all Gold requests, drop all Silver
  and Bronze requests
• Thresholds chosen based on observed arrival rates
  and service times
• Extremely efficient
• Wrong threshold gt bad response times or fewer
  requests admitted

17
Scaling Even Further

• Protocol processing overheads will saturate
  sentry resources at extremely high arrival rates
• Indiscriminate dropping of requests will occur
• Important requests may be turned away without
  even undergoing the admission control test
• Loss in revenue!
• Sentry should still be able to process each
  arriving request!
• Idea Dynamic capacity provisioning for sentry
• Pull in an additional sentry if CPU utilization
  of existing sentries exceeds a threshold (e.g.,
  90)
• Round-robin DNS to load balance among sentries

18
Talk Outline

• Motivation
• Internet data center model
• Request policing
• Cataclysm Server Platform
• Experimental results
• Summary

19
Cataclysm Server Platform

• Prototype data center
• 20 Pentium servers
• Gigabit switches
• Linux-based platform
• Sentry implemented in Layer-7 switch
• Linux module ktcpvs
• Replicated Web server applications using Apache
• Dynamic content using PHP

20
Class-based Differentiation

• Three classes of requests Gold, Silver, Bronze
• Policer successful in providing preferential
  admission to important requests

21
Threshold-based Higher Scalability

• Threshold-based processing allows the policer to
  handle upto 4 times higher arrival rate
• Single sentry can handle about 19000 req/s

22
Threshold-based Loss of Accuracy

• Higher scalability comes at a loss in accuracy of
  admission control
• Occasional violations of response time targets

23
Sentry Provisioning
24
Summary

• Cataclysm a comprehensive overload management
  technique consisting of
• Request policing
• Dynamic capacity provisioning
• SLA-based performance adaptation
• Cataclysm achieves the following
• Class-based differentiation
• Revenue maximization
• Ability to scale to extreme overloads
• More information http//lass.cs.umass.edu

25
Policing and Provisioning
26
Policing and Provisioning