Cooperative Web Caching: A Viability Study and Design Analysis

1
Cooperative Web CachingA Viability Study and
Design Analysis
Sandra G. Dykes University of Texas at San
Antonio
2
Overview

• Motivation
• Fundamental question and objectives
• Characterization of sharing
• Site selection
• Viability analysis
• Conclusions

3
Motivation

4
Why cache on the Web?

• Cache a fast, temporary store for commonly used
  items.
• Why cache?
• to reduce response time
• How do you reduce response time on the Web?
• avoid congested network routers

5
World Wide Web Protocols
Web Server
Web Browser
HTTP
HTTP


TCP
TCP
Router
Router
...
IP
IP
6
15-20 routers on a typical path

206.171.36.153 hou-core-01.inet.qwest.ne wdc-cor
e-01.inet.qwest.net wdc-core-02.inet.qwest.net n
py-core-01.inet.qwest.net 206.171.30.10 206.171.
30.14 spn-brdr-01.inet.qwest.nett
f0.pen.verio.net p1.r1.phlapa01.us.bb.verio.net
p4.r0.phlapa01.us.bb.verio.net d3.r0.mbrgpa.us.bb
.verio.net fe0-0-0.mdt1.verio.net commonwealth
1-gw.pa.verio.net www.state.pa.us
utx1-h9-1-0.tx.bb.net utx4-fe0-0-2.tx-bb.net ut
sa1.fe10-0-0-2.the.net utsa2.admin.utsa.edu
pandora.cs.utsa.edu
7
Browser and Local Proxy Caching
Web Server
Local Proxy Caches L2
Browser Caches L1
8
Proxy Cooperation (L3)
9
Can cooperative (L3) caching speed up the Web?
10
Related Work

• Cooperation designs
• Hierarchy Danzig et al. (USENIX96)
• Multcast groups Zhang et al. (Cache Wkshop,
  1997), Malpani et al. (WWW 1995)
• Push-caching Gwertzman et al. (HotOS-V,
  1995), Bestavros et al. (SPDP95)
• Directory Tewari et al.
  (ICDS99), Dykes et al. (HICSS99), Kong et al.
  (ToN, 1999)
• Quantitative comparison of designs
• Tewari et al. (ICDS99) Directory
  better than hierarchy by factor of 1.3 to 2.3
  Wolman et al. (SOSP99) Hierarchy
  better than directory by factor of 1.4
• Estimated performance gain
• Breslau et al. (Infocom99) Hit rate
  vs. requests
• Cao et al. (USITS97) Hit rate vs.
  users
• Wolman et al. (SOSP99) Hit rate
  vs. users, response time vs. users

11
Caching is faster if

• 1. the object is in the cache
• H sufficiently large (hit rate)
• 2. the cache returns the object faster than the
  Web server
• TC
• 3. cache overhead is sufficiently small
• TO sufficiently small

12
L3 Cache Speedup Flat mesh, Perfect cache
Perfect cache Perfect location knowledge, no
cache overhead, no capacity or consistency
misses
13
Dissertation Summary

• Characterize available sharing between proxy
  caches
• Estimate H3
• Understand how document cachability affects
  sharing
• Establish methods for achieving a speed
  advantage
• Evaluate site selection methods
• Estimate TC/TS for the best method.
• Analyze cooperation architectures
• Develop a taxonomy for cooperative Web caches
• Compare design approaches w.r.t. Web
  characteristics
• Determine if cooperative Web caching is viable
• Develop speedup equation and derive upper bounds
• Express speedup as a function of overhead and
  efficiency
• Explore other benefits of caching.

14
Characterization of Sharing

15
Estimating L3 Sharing

• Measured L3 parent hit rate in NLANR traces
• National Laboratory for Applied Network Research
• Operational L3 caches BO1, UC
• Real cachable documents
• Ideal all documents (upper bound for
  cachability)
• Developed techniques to correct for information
  not recorded in a trace.

16
Trace Corrections

• Repeat requests
• Cache warmup
• Sibling hit rates

17
Inferring uncachability from repeat requests

• A cursory analysis of the access logs
  revealed that there are many documents never
  being cached (at the browser and/or proxy).
  Therefore, part of the concentration of
  references seen is due to the uncachability of
  the documents. Since the access logs used in
  this analysis did not have the necessary
  information to distinguish between cachable and
  uncachable documents, no further attempt was made
  to understand the impact of cachability of
  documents.

--- Mahanti, Williamson, and Eager, IEEE
Network, May/June 2000, referring to NLANR
cache traces, among others.
18
Repeat Requests
Req. Hits Clients URL

• 14210 0 4
  m.doubleclick.net/viewad/817-grey.gif
• 4791 2 6
  ad.doubleclick.net/viewad/817-grey.gif
• 1394 0 1
  m.doubleclick.net/viewad/385809-family
• 939 0 1
  m.doubleclick.net/viewad/28902-lycoshop
• 915 21 16
  messenger.netscape.com/images/pixel.gif

19
817grey.gif
20
messenger.netscape.com/images/pixel.gif
21
What prohibits caching?

• Explicit
• cgi-bin
• query (?)
• partial content (206 status)
• cache directives in the request header
• Inferred
• cache directive in the reply header
• inferred from repeat requests

22
Cachability Analysis
1-day trace February 11, 2000
BO1 UC
Cachable 0.75 0.76 Uncachable 0.25 0.24
206 Partial content 0.00 0.00 Cgi-bin
0.01 0.00 Query 0.08 0.01 Pragma or
cache directive in request header 0.07 0.07 Inferr
ed response header (X) 0.10 0.16 Repeat
Requests 0.17 0.17 206, Cgi-bin, Query,
Pragma (CQP) 0.09 0.04 Inferred response
header (X) 0.08 0.12
23
Warming and Cachability Effects
24
Warming and Cachability Effects
25
Correcting Ideal Hit Rate for Repeat Requests

• If all documents were cachable, then most repeat
  requests would be browser hits and would not be
  seen by upper level caches.

• Not L3 hits !
• Remove repeats from the request stream

26
Hit Rate Estimates
Description
BO1 UC
27
Zipfs Law
28
Zipfs Law in Web Characterization

• ? 1.0 Glassman (1994)
• ? 0.99 Cuhna et al. (1995)
• ? 0.85 Almeida et al. (1996)
• ? 0.75 Nishikawa et al. (1998)
• ? 0.65, 0.83 Barford et al. (1994)
• ? 0.64 -- 0.83 Breslau et al. (1999)
• ? 0.74, 0.77, 0.84 Mahanti et al. (2000)

Browser traces
Server traces
Proxy traces
29
Zipf DistributionsNLANR BO1 Feb. 7, 2000
All Requests
30
Zipf Distributions
BO1 UC
a Median a Range
a Median a Range
with repeats 0.83 0.66 - 1.01
0.80 0.70 - 0.92 no repeats 0.33
0.29 - 0.46 0.22
0.19 - 0.31
M-F 0.31 0.29 - 0.35
0.21 0.19 - 0.25 Sat, Sun 0.42
0.38 - 0.46 0.29 0.27 - 0.31
W Median W Range
W Median W Range
with requests 3120 1250 - 8700
3500 1450 - 6240 no repeats
33 31 - 38 21
18 - 23
31
Sharing Observations

• Repeat requests artificially inflate ideal hit
  rates and Zipf parameters.
• Repeat requests add unnecessary traffic and
  server load.
• Hidden uncachability can be inferred from repeat
  requests.
• Simulation caches require adequate warmup.

32
Site Selection

33
Site Selection
Web caching and replication systems store object
copies at sites widely dispersed across the
Internet.

• How can clients predict which sites will be
  faster?
• Can site selection reduce the number of long
  delays?
• Is one method best under all conditions?
• What is the cache speed advantage, Tc/Ts?

34
Selection Criteria

• Static
• random, hops, bottleneck bandwidth, server
  resources
• Statistical
• median latency, median bandwidth, median
  response time (same file size only)
• Dynamic
• ping (ICMP), tcping (TCP), Squid ICP (UDP)

35
Related Research
Category DNS Routers Server-side Client-side
Examples DNS round robin IPv6 Anycast HTTP
Redirect Cisco Local Director Squid proxy
cache SPAND Smart Client Appl. Layer
Anycast Karaul, et al. Carter Crovella Sayal,
et al. this work
Predictor Variable --- Router metric
(hops) Server load RTT BW Random, Server
load Hops, Response time Server load Hops,
Response time Hops, RTT, BW Hops, RTT,
Latency Random, RTT, BW, Latency, Hybrids
36
Metric User Response Time

• T TSelect TDNS TConnect TLatency
  TRead, remaining

X
for normalized file size
37
Algorithms

• Random Select site randomly from all candidates.
• Latency Select site with lowest median latency
  in previous transfers.
• BW Select site with highest median BW in
  previous transfers.
• Probe Send tcping probe to all candidates.
  Select first to reply.
• ProbeBW Send tcping probe to the 3 sites with
  highest median BW.
• Select first to reply.
• ProbeBW2 Send probes to all candidates.
• After receiving first reply, delay 1/2 reply
  time.
• Select site with fastest prior BW among those
  who replied.

38
Methodology

• 6 client-side selection algorithms
• For each session, loop over the algorithms
• Algorithm selects a site and downloads
  designated object
• Measure each response time component
• Compute medians over sessions.
• Geographically and topologically dispersed
  servers www.state..us
• Four conditions
• Day HTML, Night HTML, Day Image, Night
  Image
• Metric is normalized response time

39
Location of Servers (S) and Clients (C)
Servers www.state..us Clients
Texas AM University (AM)

University of Houston (UH)

University of Texas at San Antonio (UTSA)
40
(No Transcript)
41
000
300
600
900
1200
1500
1800
2100
2400
42
Probability Distributions
Random
Probe
43
Observations

• Probe and ProbeBW have the best median response
  times.
• Random Latency BW ? ProbeBW2 ProbeBW ?
  Probe
• slowest --------- fastest
• There is little difference for light network
  loads or small files (10KB).
• Statistical algorithms have 2 disadvantages.
• adapt poorly to network modifications.
• require sets of calibration data for different
  file sizes and time-of-day.
• Algorithms reduce, but do not eliminate,
  pathological delays.
• TC/TS 0.20, 0.38, 0.37 Geom.
  Mean 0.30
• AM UH UTSA

44
Viability Analysis
45
Cooperation Viability and Speedup Flat mesh,
Perfect cache
Perfect location knowledge, no cache overhead,
no capacity or consistency misses
46
Viability and SpeedupFlat mesh, Imperfect
cache
does assume capacity and consistency misses are
negligible
47
Effective Hit Rate

• H3 Available sharing within a group of L2
  proxy caches. That is, the hit rate
• for a perfect cache.
• ? H3 Effective hit rate realized by the
  caching system (for imperfect caches).

48
Hit rates and Speed Advantage Estimates
0.31 0.41 0.30 0.40 (0.30 to 0.50)
L3 hit rate, Real cachability L3 hit rate,
Ideal cachability Cache speed advantage L2
Local proxy hit rate (commonly reported)
49
Upper Bound on Speedup
50
Viability ConditionThe trade-off between
overhead and efficiency
51
Viability Regions
52
Iso-speedup Condition
53
Iso-speedup CurvesReal, H2 0.4
Speedup 1.0
Speedup 1.1
Speedup 1.2
54
Reduction of Long Delays
55
Viability Conclusions

• Is cooperative Web caching viable?
• Yes, if design parameters are within overhead and
  efficiency bounds
• What are the upper bounds on speedup?
• Perfect cache, real cachability Smax 1.4
• Perfect cache, ideal cachability Smax 1.7
• Are there other performance benefits?
• reduces variance in response time and number of
  long delays
• decreases network congestion

56
Other Observations

• Its the network, not the server (generally).
• Its the overhead, not the efficiency.
• Overhead cost depends on when metadata is sent.
• Web advertisers limit caching, and the effects
  (e.g. repeat requests) have skewed previous
  research.
• TCP controls congestion by reducing offered load
  on a route.
• Web caching avoids congestion by using a better
  route.
• Improves response time for others.
• Offers a solution to UDP choking by real-time
  audio and video.

57
The real benefit of cooperation

• Congestion avoidance, not increased sharing
• more cooperation ? less congestion ? faster
  response time
• (NOT more cooperation ? more sharing ? higher
  hit rate)
• True cooperation because congestion avoidance
  improves network throughput rather than single
  user response time.

58
Contributions of Dissertation

• Viability analysis for cooperative Web caching
• Effect of repeat requests on Zipf models and
  ideal hit rates
• Techniques for correcting cache traces
• Site selection methods
• Taxonomy
• Standard terms and metrics
• Utilities httpget, tcping

59
Acknowledgements

• NSF (MII grant CDA-9633299)
• NASA/TSGC graduate fellowship
• Division of Computer Science, University of Texas
  at San Antonio
• Additional support for conference and travel
  expenses provided by USENIX and IEEE
  Communications Society.

60
Dissertation Committee

• Dr. Kay Robbins
• Dr. Clint Jeffery
• Dr. Samir Das
• Dr. Richard Sincovec
• Dr. Weining Zhang
• Dr. Larry Williams

61
END
62
(No Transcript)
63
Internet protocols
Real Time (streaming audio video)
Elastic (Web, email)
...
...
HTTP
FTP
SMTP
RTP
RTCP
TCP
UDP
ICMP
IP
64
Router Queues
Source A
Router
Source B
65
Packet Drops at Routers
Causes delay due to TCP ACK timeout and slow
start procedure.
X
Router
X
66
TCP Congestion Control
TCP reduces send rate, UDP does not.
Router
67
TCP Congestion Control vs. Web CachingA network
perspective

• Congestion control reduces the number of packets
  on a route.
• Web caching uses a different route.

68
Caching on the World Wide Web
Hierarchical
Remote Web Server
Remote Proxy Server L3 Cache
Local Proxy Server L2 Cache
Browser L1 Cache
69
Definitions

• Request
• Successful GET request
• BO1 49 (276,481 out of 563,956)
• UC 57 (397,363 out of 691,770)
• Hit
• Object returned from cache, even if HTTP
  headers are exchanged with server
• Uncachable
• Explicit cgi-bin, query, partial,
  request directives
• Inferred repeat requests (response
  directives)

70
(No Transcript)
71
(No Transcript)
72
000
300
600
900
1200
1500
1800
2100
2400
73
000
300
600
900
1200
1500
1800
2100
2400
74
(No Transcript)
75
(No Transcript)
76
(No Transcript)
77
MetricProblem Response time depends upon file
size

• User response time
• correct metric, but restricts server set because
  each server must have a copy of the same file
• Latency or bandwidth
• poor metrics because they ignore other components
  of response time
• Normalized response time
• includes all component of response time without
  restricting server set

78
File Size Normalization

• TSelect TConnect TLatency

79
Median Response Times (sec) 50 KB, M-F
9am-5pm
Overhead
Connect
Read
Total
5.6 4.4 3.0 1.3 3.7 1.1 10.1 6.3 5.0 3.5 4.6 3.8
30.3 14.2 11.5 13.5 13.8 11.2
0.00 0.00 0.02 0.08 0.06 0.08 0.00 0.00 0.02 0.07
0.05 0.06 0.00 0.00 0.00 0.20 0.17 0.19
Texas AM
Random Latency BW ProbeBW ProbeBW2 Probe Random L
atency BW ProbeBW ProbeBW2 Probe Random Latency B
W ProbeBW ProbeBW2 Probe
0.2 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1
0.7 0.2 0.2 0.2 0.2 0.2
5.2 3.5 2.3 1.1 1.4 0.9 9.6 5.9 4.4 2.3 3.7 2.9
28.4 12.6 10.7 12.5 12.7 10.7
U. Houston
UTSA
80
Measurement Procedure

• do I1 to N_measurements
• Randomly choose subset of 10 servers
• Randomly order selection algorithms
• For each algorithm
• Select server
• Download object and measure time
• Compute normalized response time
• end
• end

81
Factors affecting response time

• File size
• Network
• capacity bandwidth (slowest link)
• backbone gateway and exchange routers
• traffic load / time-of-day
• Server
• file system
• server cluster or single computer
• server load / time-of-day

82
Taxonomy and Design Analysis
83
Web Caching Taxonomy
Discovery
Dissemination
Delivery

• Fixed Cache
• Group Query
• Manual
• Automatic
• Metadata Directory
• Centralized
• Distributed

Client-initiated Server-initiated
Direct Indirect
83
84
Web Caching Designs
Projects
Discovery
Dissemination
Delivery
Direct Indirect Indirect Direct Direct
Client-initiated Client-initiated Client-initiate
d Server-initiated Client-initiated
Proxy server cache Hierarchical Harvest,
Squid (NLANR) Multicast groups AWC
Zhang, et al. Malpani, et al. Push
caching GPC Gwertzman Seltzer
DDD Bestavros, et.al. Local directory
Tewari, Dahlin ... SDP Dykes, Jeffery,
Das
Fixed cache Group query Manual Group query
Auto Directory Centralized Directory
Distributed
85
Design Recommendations

• Flat mesh organization for delivery.
• Local metadata directory for discovery.
• Propagate most metadata during quiescent periods.
  Supplement with small updates.

• miss time TC
• TO independent of number of requests
• TO independent of number of objects, but
  introduces ?