Logistical Networking

1
Logistical Networking

• Alessandro Bassi
• LoCI lab, University of Tennessee
• On leave at the RESO team / ENS Lyon, France
• "Grillification" des applications - 1er Avril 2003

2
What is all about
3
Logistical Networking

• Logistical Networking global scheduling and
  optimisation of data movement, storage and
  computation.
• High-perf networking capabilities enabled
  distributed computing innovations such as
  explicit control of computation resources (I.e.,
  Computational Grid)
• Application driven storage control -gt less common

4
Logistical Networking

• Technology for shared network storage that can
  scale in terms of
• the size of the user community,
• the aggregate quantity of storage that can be
  allocated, and
• the breadth of distribution of service nodes
  across network borders
• Parallels layered IP networking stack

5
What can it do for you?
6
How can you transfer big files?
7
Moving files quickly
8
Moving files quicker
9
Logistical Networking Scenarios
Sender
Receiver
IBP
Network
Sender
IBP
Receiver
Network
Sender
IBP
Receiver
IBP
IBP
Sender
IBP
Receiver
IBP
IBP
10
The Big Picture
11
Internet Backplane Protocol

• Middleware for managing and using remote storage
  through allocation of primitive byte arrays
  available to clients
• Byte arrays are not blocks (more abstract)
• Network capabilities (primitive security)
• Variable extents

• Byte arrays are not files (weaker semantics)
• Size duration are limited
• Volatile allocations
• Best effort reliability and availability
• No directory structure, accounting
• No caching, replication

12
Ex-Node

• A unix inode aggregates disk blocks on a single
  disk volume to implement a file
• An external Node (exNode) aggregates storage
  allocations on the internet to implement a
  network file

13
Active file management

• Active Services
• Active Probing to Maintain Fault Tolerance
• Lease Renewal
• Optimal fragmentation
• Asynchronous Transfer Management

14
L-Bone

• The L-Bone is an LDAP-based directory of
  available IBP depots.
• The client library allows users to query the
  L-Bone for depots that meet certain requirements

15
Public L-Bone today
16
Logistical Runtime System

• Aggregation for
• Capacity
• Performance (striping)
• More performance (caching)
• Reliability (replication)
• More reliability (ECC)
• Logistical purposes (routing)

17
LoRS - basic primitives

• Upload Creates a network file from local data
• Download Get bytes from a network file
• Augment Add more replicas to a network file
• Trim Remove replicas from a network file
• Stat Get information about the network file
• Refresh Alter the time limits of the IBP buffer

18
Conclusions

• The Internet Backplane Protocol provides a fabric
  for network state management
• Scalable and ubiquitous sharing are enabled by
  weak semantics minimal accounting
• The exNode serialization provide a mobile
  abstraction of storage aggregation
• The l-bone maintains informations about the state
  of connected depots
• Directions of development (upwards, downwards,
  deployment)
• To know more loci.cs.utk.edu
• Alessandro.Bassi_at_ens-lyon.fr