EnsemBlue: Integrating Distributed Storage and Consumer Electronics

1
EnsemBlue Integrating Distributed Storage and
Consumer Electronics

• Daniel Peek and Jason Flinn
• University of Michigan

2
Bringing Distributed Systems Home

• Distributed systems ignore Consumer
  Electronic Devices (CEDs)
• Much user interaction is through CEDs
• Integrate CEDs into distributed systems

3
CED Integration Difficult

• Application-specific resources
• Heterogeneous interfaces
• Closed platforms
• Cannot execute our code

4
Today
5
EnsemBlue

• Hypothesis Distributed file systems can help
  manage CEDs and multimedia

My Data
My Devices
6
EnsemBlue

• Based on BlueFS Nightingale 04
• Client-server design
• Single namespace
• Supports mobile clients
• Designed for small group of users

File Server
7
Outline

• Motivation
• Device-Specific Namespaces
• Device-Specific Functionality
• Disconnected Collections of Devices
• Evaluation
• Conclusion

8
Connecting CEDs
Distributed File System
Device-Specific Protocol
DFS Protocol?
Device-Specific Protocol

• CEDs cannot perform DFS protocol

• Attach computer speaks for CED

• Works with disconnected clients

9
Device-Specific Namespaces

• User prefers one organization
• CED requires another organization

10
Device-Specific Namespaces

• Where to store the mapping?

• Cannot store in DFS due to partial caching

• Make CEDs self-describing

11
Receipts

• What should mapping look like?

dpeek/classical/song.mp3
/iPod/f27/ABCD.mp3
Object 1.123.46F2

• Path lookups may fail with partial caching

• Receipts bypass path resolution

12
Device-Specific Execution

• Updating indexes
• Transcoding
• Type-specific caching
• Organizers

13
Device-Specific Execution

• Cannot execute on CED
• Leverage general computers
• Take action when files change
• Problem Need notification of file changes

14
Notification Via Persistent Queries

• Dont need new mechanism
• Leverage cache consistency mechanism
• Structure notifications as file system object
• Robust to crash
• Handles disconnected operation

updates
truncate
.mp3
read
process
15
Persistent Queries

• Example M4A to MP3 transcoder

File Server
M4A
M4A
MP3
M4A to MP3 Transcoder
.m4a
MP3
M4A
MP3
.m4a
16
Disconnected Devices

• Server gives safety, availability, consistency
• Server mediates communication

File Server
17
Disconnected Devices

• Disconnected devices cannot interact!

File Server
18
Disconnected Devices

• Peer-to-Peer better supports mobility
• CEDs store and forward updates

File Server
19
Ensembles

• Ensembles can avoid these problems
• Select a pseudo file server
• Consistent view of all files
• Improved data access
• Update propagation

Pseudo File Server
20
Outline

• Motivation
• Device-Specific Namespaces
• Device-Specific Functionality
• Disconnected Collections of Devices
• Evaluation
• Conclusion

21
Microbenchmarks

• Persistent query execution is fast
• lt200 ms for 1,000s of matches, GBs of data
• Persistent queries have low overhead
• Negligible even for 10,000 queries
• Ensemble formation as fast as 20 ms
• Proportional to data transfer size

22
Organizing My Photos
File Server
.jpg
Photo Organizer
iCal
23
Managing My Music
File Server
MP3
DB
MP3
DB
MP3
DB
.mp3
iPod DB Updater
MP3
DB
MP3
24
Conclusion

• Added CEDs to distributed systems
• Device-specific data organization
• Device-specific functionality
• Supports TiVo, iPod, digital cameras, media
  players, cell phones, PDAs

25
Questions
?