A framework for implementing IO-bound maintenance applications

1
A framework for implementing IO-bound maintenance
applications

• Eno Thereska

2
Disk maintenance applications

• Lots of disk maintenance apps
• data protection (backup)
• storage optimization (defrag, load bal.)
• caching (write-backs)
• Important for system robustness
• Background activities
• should eventually complete
• ideally without interfering with primary apps

3
Current approaches

• Implement maintenance application as foreground
  application
• competes for bandwidth or off-hours only

4
Current approaches

• Trickle maintenance activity periodically
• lost opportunities due to inadequate scheduling
  decisions

5
Real support for background applications

• Push maintenance activities in the background
• priorities and explicit support for them
• APIs allow application expressiveness
• Storage subsystem does the scheduling
• using idle time, if there is any
• using otherwise-wasted rotational latency in a
  busy system

6
Outline

• Motivation and overview
• The freeblock subsystem
• Background application interfaces
• Example applications
• Conclusions

7
The freeblock subsystem

• Disk scheduling subsystem supporting new APIs
  with explicit background requests
• Finds time for background activities
• by detecting idle time (short and long bursts)
• by utilizing otherwise-wasted rotational latency
  in a busy system

8
After reading blue sector
After BLUE read
9
Red request scheduled next
After BLUE read
10
Seek to Reds track
After BLUE read
Seek for RED
11
Wait for Red sector to reach head
After BLUE read
Seek for RED
Rotational latency
12
Read Red sector
After BLUE read
Seek for RED
Rotational latency
After RED read
13
Traditional service time components
After BLUE read
Seek for RED
Rotational latency
After RED read

• Rotational latency is wasted time

14
Rotational latency gap utilization
After BLUE read
15
Seek to Third track
After BLUE read
Seek to Third
SEEK
16
Free transfer
After BLUE read
Seek to Third
Free transfer
SEEK
FREE TRANSFER
17
Seek to Reds track
After BLUE read
Seek to Third
Seek to RED
Free transfer
SEEK
SEEK
FREE TRANSFER
18
Read Red sector
After BLUE read
Seek to Third
Seek to RED
After RED read
Free transfer
SEEK
SEEK
FREE TRANSFER
19
Steady background I/O progress
40
from idle time
from rotational gaps
35
30
25
Free MB/s
20
15
10
5
0
0
10
20
30
40
50
60
70
80
90
100
disk utilization by foreground (random 4KB)
reads/writes
20
The freeblock subsystem (cont)

• Implemented in FreeBSD
• Efficient scheduling
• low CPU and memory utilizations
• Minimal impact on foreground workloads
• lt 2
• See refs for more details

21
Outline

• Motivation and overview
• The freeblock subsystem
• Background application interfaces
• Example applications
• Conclusions

22
Application programming interface (API) goals

• Work exposed but done opportunistically
• all disk accesses are asynchronous
• Minimized memory-induced constraints
• late binding of memory buffers
• late locking of memory buffers
• Block size can be application-specific
• Support for speculative tasks
• Support for rate control

23
API description task registration
application
fb_read (addr_range, blksize,) fb_write
(addr_range, blksize,)
Foreground
Background
background scheduler
foreground scheduler
24
API description task completion
application
callback_fn (addr, buffer, flag, )
Background
Foreground
foreground scheduler
background scheduler
25
API description late locking of buffers
application
buffer getbuffer_fn (addr, )
Background
Foreground
foreground scheduler
background scheduler
26
API description aborting/promoting tasks
application

fb_abort (addr_range, ) fb_promote (addr_range,
)
Foreground
Background
background scheduler
foreground scheduler
27
Complete API
28
Designing disk maintenance applications

• APIs talk in terms of logical blocks (LBNs)
• Some applications need structured version
• as presented by file system or database
• Example consistency issues
• application wants to read file foo
• registers task for inodes blocks
• by time blocks read, file may not exist anymore!

29
Designing disk maintenance applications

• Application does not care about structure
• scrubbing, data migration, array reconstruction
• Coordinate with file system/database
• cache write-backs, LFS cleaner, index generation
• Utilize snapshots
• backup, background fsck

30
Outline

• Motivation and overview
• The freeblock subsystem
• Background application interfaces
• Example applications
• Conclusions

31
Example 1 Physical backup

• Backup done using snapshots

backup application
getblks()
sys_fb_read()
sys_fb_getrecord()
snapshot subsystem (in FS)
freeblock subsystem
32
Example 1 Physical backup

• Experimental setup
• 18 GB Seagate Cheetah 36ES
• FreeBSD in-kernel implementation
• PIII with 384MB of RAM
• 3 benchmarks used Synthetic, TPC-C, Postmark
• snapshot includes 12GB of disk
• GOAL read whole snapshot for free

33
Backup completed for free
90
80
70
60
lt 2 impact on foreground workload
50
40
Backup time (mins)
30
20
10
0
Idle system
Synthetic
TPC-C
Postmark
34
Example 2 Cache write-backs

• Must flush dirty buffers
• for space reclamation
• for persistence (if memory is not NVRAM)
• Simple cache manager extensions
• fb_write(dirty_buffer,)
• getbuffer_fn(dirty_buffer,)
• fb_promote(dirty_buffer,)
• fb_abort(dirty_buffer,)

35
Example 2 Cache write-backs

• Experimental setup
• 18 GB Seagate Cheetah 36ES
• PIII with 384MB of RAM
• controlled experiments with synthetic workload
• benchmarks (same as used before) in FreeBSD
• syncer daemon wakes up every 1 sec and flushes
  entries that have been dirty gt 30secs
• GOAL write back dirty buffers for free

36
Foreground readwrite has impact
100
100
80
80
60
60
improvement in avg. resp. time
dirty buffers cleaned for free
40
40
20
20
0
0
0
12
11
21
0
12
11
21
read-write ratio
read-write ratio
37
95 of NVRAM cleaned for free
100
100
80
80
60
60
dirty buffers cleaned for free
improvement in avg. resp. time
40
40
20
20
0
0
LRUSyncer
LRU only
LRUSyncer
LRU only
38
10-20 improvement in overall perf.
50
50
40
40
30
30
dirty buffers cleaned for free
improvement in app. throughput
20
20
10
10
0
0
Synthetic TPC-C Postmark
Synthetic TPC-C Postmark
39
Example 3 Layout reorganizer

• Layout reorganization improves access latencies
• defragmentation is a type of reorganization
• typical example of background activity
• Our experiment
• disk used is 18GB
• we want to defrag up to 20 of it
• goal defrag for free

40
Disk Layout Reorganized for Free!
600
Random
500
Circular
400
Track Shuffle
300
Reorganization time (mins)
200
100
0
1
10
20
1
10
20
8MB
64MB
Reorganizer buffer size (MB)
41
Other maintenance applications

• Virus scanner
• LFS cleaner
• Disk scrubber
• Data mining
• Data migration

42
Summary

• Framework for building background storage
  applications
• Asynchronous interfaces
• applications describe what they need
• storage subsystem satisfies their needs
• Works well for real applications
• http//www.pdl.cmu.edu/Freeblock/