Awareness Services for Digital Libraries

1
Awareness Services for Digital Libraries

• Arturo Crespo
• Hector Garcia-Molina
• Stanford University

2
Awareness Services for Digital Libraries

• Digital library repository
• Data store
• Other components
• Indexers
• Name manager
• Replica manager
• Etc

3
Data Stores and Clients
DB Tech Reports
DB Indexer
AI Tech Reports
CS Indexer
HCI Tech Reports
Data Stores
Clients
4
Data Store Services

• Object access
• Via a handle
• Object awareness
• Clients must be aware of changes at the store

5
A Case Study CS-TR and SIFT

• SIFT a selective dissemination service
• CS-TR A digital library of technical reports
  from about 50 universities
• Awareness based on timestamps
• Problems
• File system timestamps
• Application timestamps
• Deletions

6
Contributions

• Survey of the spectrum of awareness options
• Advantages and disadvantages of each one
• All mechanisms can be capture by a single
  algorithm the UNI-AWARE algorithm
• Enhancements for signature-based schemes
• Reduced computation
• Reduced communication costs

7
Related Work

• Database replica maintenance
• Remote file comparison
• Deployment of programs over the network

8
The Client-store Design Space

• Push vs. Pull
• Statefull versus stateless stores and clients
• Cognizant clients and sources
• Number of clients per data store

9
The UNI-AWARE Algorithm

• A unified algorithm that covers known schemes
• Snapshot algorithm
• Timestamps and versions
• Logs
• Triggers
• Signatures
• Algorithm is tailored to a specific scheme
  through the definition of custom functions

10
UNI-AWARE Signature Algorithm

• Signature a token associated with each document
  that has a high probability of being unique and
  changes when the content of the object changes
• Example CRC, checksums
• Advantages
• Robust as it does not require metadata
  maintenance
• Easy to manage consistently when store fails or
  object migrates

11
UNI-AWARE Signature Algorithm
All signatures transferred
Data Store
Client
Document
Signature
Request Documents
12
DIST-UNI-AWARE Algorithm

• Objective reduce amount of data exchanged
  between data store and clients
• DIST-UNI-AWARE
• Unified algorithm that can be tailored to
  different schemes
• Hierarchical signatures
• Hierarchical timestamps

13
DIST-UNI-AWARE
Signatures of Buckets transferred
Data Store
Client
Request more Signatures
Request Documents
Document
Signature
14
Advantages of Signature Algorithms

• Support the push and pull models
• No need for reliable storage of additional data
  structures if signatures are lost or corrupted,
  they can be recomputed
• Efficient in usage of network resources, clients
  and data stores
• Scales well in number of clients and documents

15
DIST-UNI-AWARE Performance

• Performance depends on number of changes
• No changes only one round is required
• Single change log2n rounds
• 2 changes log2n rounds, but twice as much data
• Eventually, DIST-UNI-AWARE starts behaving worse
  than UNI-AWARE

16
DIST-UNI-AWARE Enhancements

• Increase group split factor
• Client sends additional information at split time
• Clustering of changed objects

17
Conclusions

• Awareness mechanism for digital libraries
• Separation of storage functionality and other
  services
• Awareness schemes must be resilient to computer
  environment changes and bugs
• UNI-AWARE and DIST-UNI-AWARE

18
Awareness Services for Digital Libraries
Arturo Crespo Hector Garcia-Molina Stanford
University