Exploiting Gray-Box Knowledge of Buffer Cache Management

1
Exploiting Gray-Box Knowledge of Buffer Cache
Management

• Nathan C. Burnett, John Bent,
• Andrea C. Arpaci-Dusseau,
• Remzi H. Arpaci-Dusseau
• University of Wisconsin - Madison
• Department of Computer Sciences

2
Knowledge is Power

• OS usually treated as a black box
• Implies no knowledge of OS internals
• OS can be viewed as a gray-box
• We have basic knowledge of the OS
• File caching
• Process scheduling
• Gray-box knowledge can be leveraged
• To discover more detailed knowledge
• To implement OS-like services from user-space

3
Knowledge is Power

• Cache policy impacts I/O performance
• Programmers can use knowledge of cache state to
  improve overall performance
• Cache policy is usually hidden
• Documentation can be old, vague or incorrect
• Source code may not be available
• Often no interface for learning cache state

4
Fingerprinting

• Use microbenchmarking techniques to discover
  algorithms and policies
• Dust - Fingerprints buffer cache policies
• Correctly identifies many different policies
• Requires no kernel modification
• Portable across platforms

5
This Talk

• Dust
• Fingerprints of Real Systems
• Exploiting Gray-Box Knowledge
• Cache-Aware Web Server
• Conclusions Future Work

6
Dust

• Fingerprints the buffer cache
• Determines cache size
• Determines cache policy
• Determines cache history usage
• Uses only standard system calls
• read/open/close/seek

7
Approach

• Cache policies often use access order, recency,
  frequency
• Need to find an access pattern that will
  differentiate FIFO, LRU, LFU
• Explore in simulation
• Keeps track of cache contents
• Doesnt actually store the data
• Two possible access times (in cache or not)

8
Approach

• Move the cache to a known state
• Sets recency
• Sets access order
• Sets frequency
• Cause part of test data to be evicted
• Sample data to determine cache state

9
Detecting FIFO Replacement
Test Region
Eviction Region

• Test Region slightly smaller than the cache
• One sequential scan sets access order
• Eviction Scan evicts half of the Test Region

10
Detecting FIFO Replacement
Newer Pages
Older Pages
FIFO will give the later part of the file
priority in the cache
11
Detecting FIFO Replacement
Out of Cache
In Cache

• FIFO evicts the first half of the test region

12
Detecting LRU Replacement
Test Region
Eviction Region
Right Pointer
Left Pointer

• Need to differentiate from FIFO
• Divide Test Region into ten stripes
• Reads alternate between left and right pointers

13
Detecting LRU Replacement
LRU will give priority to the 2nd and 4th
quarters of the test region
14
Detecting LRU Replacement

• LRU evicts the first and third quarter of the
  test region

15
Detecting LFU Replacement
Test Region
Eviction Region
2
3
4
5
6
6
5
4
3
2
7
Right Pointer
Left Pointer

• Read stripes near the middle more
• Read stripes near the ends less
• Read eviction region more than any stripe

16
Detecting LFU Replacement
LFU gives priority to the center of the test
region
17
Detecting LFU Replacement

• LFU evicts the outermost stripes.
• Two stripes partially evicted.

18
Detecting Clock Replacement

• Clock often used in place of LRU
• Two pieces of initial state
• Hand Position
• Reference Bits
• Hand position is irrelevant circular queue
• Dust must control for reference bits
• Reference bits affect the order of replacement

19
Detecting Clock Replacement

• Uniform reference bits

• Random reference bits

20
Clock - Random Reference Bits

• Initial Sequential Scan
• Read pattern does not change cache state
• First part of left and right halves are evicted

21
Clock - Uniform Reference Bits

• First part of test region is evicted
• Degenerates to FIFO

22
Clock - Controlling Reference Bits

• Two fingerprints for Clock
• Ability to produce both will imply Clock
• Need a way to selectively set reference bits

23
Clock - Controlling Reference Bits

• To set all reference bits
• Read in enough data to almost fill the cache
• Touch every page
• To clear all reference bits
• Read small amount of new data to cause
  replacement
• Clock hand sweeps entire cache, resetting all
  refbits
• To randomly set half of the bits
• Randomly touch half of the pages in the cache

24
Dust

• Differentiates policies based on access order,
  recency and frequency
• Determines cache size
• Identifies more complex policies
• Segmented FIFO, Random
• Identifies history-based policies
• LRU-k, 2-Queue
• Robust to errors in cache size estimate

25
This Talk

• Dust
• Fingerprints of Real Systems
• Exploiting Gray-Box Knowledge
• A Cache-Aware Web Server
• Conclusions Future Work

26
Fingerprinting Real Systems

• Data is noisy
• Policies usually more complex
• Buffer Cache/VM Integration
• Cache size might be changing
• Timing Precision

27
NetBSD 1.5

• Clock Fingerprint is identical to this
• Conclusion LRU

28
Linux 2.2.19

• Very noise but looks like LRU

29
Linux 2.2.19

• With uniform reference bits, looks like FIFO
• Conclusion Clock

30
Linux 2.4.14

• Shows the two humps of LRU
• Shows end hump of LFU
• Is LRU with page aging

31
This Talk

• Dust
• Fingerprints of Real Systems
• Exploiting Gray-Box Knowledge
• A Cache-Aware Web Server
• Conclusions Future Work

32
Algorithmic Mirroring

• Observe inputs presented to the kernel
• We have knowledge of kernel algorithms
• Model internal state of the kernel
• Only model relevant state, not everything
• Can be an approximation
• Use model to make application-level decisions

33
A Cache-Aware Application

• NeST - Network Storage Technology
• Software based storage appliance
• Supports HTTP, NFS, FTP, GridFTP, Chirp
• Allows configurable number of requests to be
  serviced concurrently
• Any requests beyond that number are queued

34
Cache-Aware NeST

• Takes policy size discovered by Dust
• Maintains algorithmic mirror of the cache
• Updates mirror on each request
• No double buffering
• May not be a perfect mirror
• Approximate shortest-job-first scheduling by
  scheduling in-cache-first
• Reduce latency by approximating SJF
• Improve throughput by reducing disk reads

35
Improved Response Time

• Dramatic improvement in response time
• Robust to inaccuracies in cache estimate

36
Conclusions

• Fingerprinting
• Discovers OS algorithms and policies
• Dust
• Discovers buffer cache policies
• Algorithmic Mirroring
• Keep track of kernel state in user-space
• Use this information to improve performance
• Cache-Aware NeST
• Uses mirroring to improved HTTP performance

37
Future Work

• On-line, adaptive detection of cache policy
• Policy manipulation
• Make other applications cache aware
• Databases
• File servers (ftp, NFS, etc.)
• Fingerprint other OS components