IB in the Wide Area

1
IB in the Wide Area

• How can IB help solve large data problems in the
  transport arena

2
Enhanced Capabilities

• Near real-time access of operational data across
  the global filesystem
• Rapid access to data means faster, more
  comprehensive, more accurate analysis
• Reach back/forward to any data in enterprise to
  enhance change/pattern, IW and critical node
  analysis
• Enables Real-time (Within Seconds) Ingestion of
  huge data sets
• Enhanced situational awareness
• Enables dynamic updates to users and analysts
• Search within theater data streams
• Upgraded local storage/caching in reduced
  footprint

3

1. Global Situational Awareness requires dynamic
   access from very large, distributed data centers
   
2. Latest imagery streams into global services
3. OTD and Commercial applications used to visualize
   new imagery
4. Cross-node dynamic data sharing for collaboration

4
Why do we need a scalable Global File System

• Must support 100s of users working on thousands
  of large (GB and TB) file sets
• There is nothing like this today with the needed
  information assurance features
• The closest is what some in the HPC are doing on
  clusters
• Must work on multiple transport and storage
  protocols to get to end users
• IP, IB, FC, NFS, CIFS, etc
• Global Must work over long distances and
  multiple data centers

5
Storage-wise

• The earlier truths drive demand for the
  following capabilities
• Capability disaster recovery
• Data replication for disaster recovery
• Capability performance
• Geographic locality of data for performance
  reasons
• Caching, prefetching
• Capability wide area accessability
• All clients have access to the same datasets (/-
  security reqmts)
• In a nutshell, youd like to be able to inject
  raw information into the system from anywhere,
  and youd like to provide access to both the raw
  information and processed data to consumers
  located anywhere. And, you want to be sure the
  data is protected and continuously accessible.
• Thats it!

6
Storage-wise

• This suggests, in turn
• a data store which is globally accessible
• to facilitate availability and disaster recovery
• a globally coherent, application level name space
• That means that a file identifier used by an
  application located anywhere in the system will
  always resolve to the same underlying data. This
  is important to enable global access to data, to
  simplify disaster recovery, to support highly
  available systems and for a host of other
  reasons.
• The idea is to make providers and consumers of
  information truly portable

7

• There are at least three ways to achieve a
  globally coherent file-level namespace
• Brute force Create a single, globally flat file
  system
• Create a single global block-level name space,
  then force a strong binding to a consistent
  file-level name space among a set of distributed,
  federated filesystems
• Create a consistent file-level name space among a
  set of distributed, federated filesystems

8
A globally coherent file-level namespace

site 1

site 1

site n
site 1
site n
site n
File I/O
file system
f/s 1
f/s n
f/s 1
f/s n
block I/O
BV
BV
Global file system
Federated file system
Block approach
As long as coherence is maintained between the
federated filesystems, and as long as each f/s
has access to the global data pool, there is no
need to enforce a strict binding between
block-level I/O and file-level I/O. Coherence at
the block I/O level is neither necessary nor
sufficient.
9
(No Transcript)
10
(No Transcript)
11
An Idea of a Wide and Distributed Data Center
Construct
Core Node Components
Other Users
Network Storage Controller
InfiniBand WAN GATEWAY
STORAGE ARRAY (Petabytes)
ENCRYPTOR (10 Gbps)
LAN with Local Servers, Apps, Nets
InfiniBand WAN OC-192 (10 Gbps) Core
VLH
12
(No Transcript)