Load Shedding Techniques for Data Stream Systems

1
Load Shedding Techniques for Data Stream Systems

• Brian BabcockMayur DatarRajeev
  MotwaniStanford University

2
Differences from Previous Talk

• Our focus Aggregation queries
• No quality of service specifications
• Instead, focus on accuracy of query answers
• Compensate for dropped data by scaling answers
• Random drops only (no semantic drops)

3
Problem Setting
Q1
Q2
Q3
S
S
S
?
?
?
?
R
S1
S2
4
Inputs to the Problem
Q1
Q2
Q3
S
S
S
?
?
?
?
R
S1
S2
5
Load Shedding via Random Drops
(time, selectivity)
(t3, s3)
Load rt1 rs1t2 rs1s2t3
(t2, s2)
Sampling Rate p
(t1, s1)
Need Load 1
Stream Rate r
6
Problem Statement

• Relative error is metric of choice Estimate -
  Actual
• Actual
• Goal Minimize the maximum relative error across
  queries, subject to Load 1
• Want low error with high probability

7
Relating Load Shedding and Error

• Equation derived from Hoeffding bounds
• Constant Ci depends on
• Variance of aggregated attribute
• Sliding window size

8
Calculate Ratio of Sampling Rates

• Minimize maximum relative error ? Equal relative
  error across queries
• Express all sampling rates in terms of common
  variable ?

9
Placing Load Shedders
Target .8?
Target.6?
S
S
?
?
Sampling Rate .75 .6? /.8?
?
Sampling Rate .8?
10
Conclusion

• Load shedding helps cope with bursts
• Minimizing relative error is natural objective
  for aggregate queries
• Algorithm for load shedding
• Relate target sampling rates for all queries
• Place random drop operators based on target
  sampling rates
• Adjust sampling rates to achieve desired load

11
Thanks for listening!

• Questions?

12
Choosing Target Sampling Rates
13
Measuring Inaccuracy
Scale answer by 1/(p1p2)
S3
Sampling Rate p2
?2

• Key point Product of sampling rates determines
  quality of approximate answer

Sampling Rate p1
?1