A Highly Available Cluster Storage System Using Scavenging - PowerPoint PPT Presentation

1 / 21
About This Presentation
Title:

A Highly Available Cluster Storage System Using Scavenging

Description:

High Availability and Performance Computing Workshop (HAPCW' ... Scavenger Registration. A Highly Available Cluster Storage System Using Scavenging, HAPCW'2004 ... – PowerPoint PPT presentation

Number of Views:157
Avg rating:3.0/5.0
Slides: 22
Provided by: david3103
Category:

less

Transcript and Presenter's Notes

Title: A Highly Available Cluster Storage System Using Scavenging


1
A Highly Available Cluster Storage System Using
Scavenging
  • High Availability and Performance Computing
    Workshop (HAPCW'04), held with
  • 2004 Los Alamos Computer Science Institute
    Symposium (LACSI2004)

Xubin (Ben) He, Li Ou hexb, lou21_at_tntech.edu St
orage Technology Architecture Research(STAR)
Lab Electrical and Computer Engineering
Department
Stephen L. Scott, Christian Engelmann scottsl,
engelmannc_at_ornl.gov Computer Science and
Mathematics Division
2
Outline
  • Introduction
  • Highly Available Metadata Management
  • Metadata Management using Bloom Filters
  • Conclusions and Future Work

3
Motivations
  • Data intensive scientific applications ? high
    performance computing.
  • Distributed storage systems high performance
    storae, wide area mass storage, cluster storage.
  • Excellent performance, parallel support
    ??administration costs, central points of failure
    and control

4
Existing work
  • HPSS
  • Peer-to-Peer Storage
  • Scavenging Vazhkudai

5
  • Introduction
  • Highly Available Metadata Management
  • Metadata Management using Bloom Filters
  • Conclusions and Future Work

6
User Data vs. Metadata
Metadata manager
Storage server
Client
7
Distributed Metadata Management
Client Access
Cluster Storage Management
Scavenge Manager Unified Interface
Metadata Management Layer
Scavenged Storage Morsels
8
High Availability Scheme 1 (Pure P2P)
Peer-To-Peer Network
Manager
Metadata
9
High Availability Scheme 2 (active/hot-standby)
Peer-To-Peer Network
Active Manager
Metadata
Hot-Standby Manager
10
High Availability Scheme 3(Partitioning)
Master/Load Redirector
Network
Manager
Metadata
11
High Availability Scheme 4(leader/helper)
Network
Metadata
Helper
Leader
12
Group Communications
  • Peer-to-Peer distributed control
  • Reliable broadcast, Atomic Broadcast
  • Atomic transactions guarantee metadata integrity

13
  • Introduction
  • Highly Available Metadata Management
  • Metadata Management using Bloom Filters
  • Conclusions and Future Work

14
Bloom Filter
  • A Bloom filter is a fast and efficient method for
    representing a set Aa1,a2,,an of n elements
    to support membership queries.

Element a
h1(a)P1
h2(a)P2
m bits
h3(a)P3
h4(a)P4
15
Metadata Management using Bloom Filters
File queries
Scavenge Manager Unified Interface
Update
hit
Hashing and LRU Cache
miss
Bloom Filter Array
hit
miss
Multicast the queries
16
  • Introduction
  • Highly Available Metadata Management
  • Metadata Management using Bloom Filters
  • Conclusions and Future Work

17
Conclusions
  • Investigate Availability issues Scavenged Storage
    Systems, propose 4 solutions for maintaining
    multiple metadata managers
  • P2P
  • Active/hot-standby
  • Partitioning
  • Leader/Helper
  • Speed up the metadata searching
  • Bloom Filters

18
Future Work
  • Comparing the proposed 4 schemes
  • Scalability
  • Metadata Cache

19
Acknowledgements
  • Research Office and Center for Manufacturing
    Research, Tennessee Technological University
  • Ralph E. Powe Junior Faculty Enhancement Award by
    Oak Ridge Associated Universities (ORAU).
  • Mathematics, Information and Computational
    Sciences Office, Office of Advanced Scientific
    Computing Research, Office of Science, U. S.
    Department of Energy.

20
Questions and Comments?
21
A Highly Available Cluster Storage System Using
Scavenging
  • High Availability and Performance Computing
    Workshop (HAPCW'04), held with
  • 2004 Los Alamos Computer Science Institute
    Symposium (LACSI2004)

Xubin (Ben) He, Li Ou hexb, lou21_at_tntech.edu St
orage Technology Architecture Research(STAR)
Lab Electrical and Computer Engineering
Department
Stephen Scott, Christian Engelmann scottsl,
engelmannc_at_ornl.gov Computer Science and
Mathematics Division
Write a Comment
User Comments (0)
About PowerShow.com