Optimizing Distributed Application Performance Using Logistical Networking

1
Optimizing Distributed Application Performance
Using Logistical Networking

• Micah Beck Jack DongarraJames S. Plank Rich
  Wolski
• Computer Science Dept.
• University of Tennessee, Knoxville

2
Logistical Networking

• The use of storage resources as part of the
  networks communicative infrastructure.
• Examples
• mail, news, FTP, HTTP
• file and database caches, replication
• remote buffer managment
• distributed shared state

3
The Logistical Computing Internetworking
(LoCI) Hypothesis

• If network resources (distributed storage,
  communication and computation) can be predictably
  allocated and flexibly coscheduled,
• Then advanced applications can be implemented
  with higher performance and/or lower overall use
  of communication, computational, and storage
  resources.

4
Management of State in the Network Motivation

• The design of distributed systems for performance
  and modularity requires management of state
• The TCP/IP protocols implement data transmission
  in a stateless network.
• State management functions are currently pushed
  up to the application layer using
  application-specific protocols.

5
The Internet Backplane Protocol

• The network is a commons used by a community to
  enable communication.
• Data transmission is currently the only resource
  held in common.
• IBP adds storage to the commons.
• Substantial storage resources are owned by the
  network and allocated to endpoints.

6
IBP Client API Allocation

• IBP_allocate(char host, int size, IBP_attributes
  attr)
• Open allocation is key to networking
• Attributes key to avoiding denial-of-use
• time limitation
• volatile (server can revoke)
• Circular, FIFO

7
Dimensions in Communication

• The Internet allows spatial flexibility through
  transmission and routing
• no temporal flexibility as quick as possible
• Storage allows temporal flexibility through
  storage for retrieval at later time(s)
• no spatial flexibility the bits stay put
• IBP allows flexibility in both dimensions

8
Dimensions in Communication
Networking (spatial)
Storage (temporal)
Logistical Networking
9
Example NetSolve State Management
Client
A,B
Server 1

• The Problem NetSolve calls are functional
• Excessive data transfers

F
A
Client
A,B
Server 2
F
A
For example
Client
A F(A, B) A F(A, B)
10
Caching
Dependence Flow
A,B
Client
Client
Server 1
A
A,B
Server 1
B
F
A
F
B
A
B
IBP Cache
Server 2
A,B
Server 2
A
F
F
A
A
Client
Client
11
Logistical NetworkingResearch Issues

• Allocation policy
• Security
• Prediction
• Co-scheduling
• Network Routing
• Dynamic control
• Peering

• Reliability
• Disaster recovery
• Resource discovery
• Garbage collection
• Cost recovery
• Quality of Service
• Performance