A Common API for Structured Peer-to-Peer Overlays

1
A Common API for Structured Peer-to-Peer Overlays

• Frank Dabek, Ben Y. Zhao,Peter Druschel, Ion
  Stoica

2
Structured Peer-to-Peer Overlay

• They are
• Scalable, self-organizing overlay networks
• Provide routing to location-independent names
• Examples CAN, Chord, Pastry, Tapestry,
• Basic operation
• Large sparse namespace N(integers 02128 or
  02160)
• Nodes in overlay network have nodeIds ? N
• Given k ? N, a deterministic function maps k to
  its root node (a live node in the network)
• route(msg, k) delivers msg to root(k)

3
Current Progress

• Lots of applications built on top
• File systems, archival backup
• Application level multicast
• Routing for anonymity, attack resilience
• But do we really understand them?
• What is the core functionality that applications
  leverage from them?
• What are the strengths and weaknesses of each
  protocol? How can they be exploited by
  applications?
• How can we build new protocols customized to our
  future needs?

4
Our Goals

• Protocol comparison
• Compare and contrast protocol semantics
• Identify basic commonalities
• Isolate and understand differences
• Towards a common API
• Easily supportable by old and new protocols
• Enables application portability between protocols
• Enables common benchmarks
• Provides a framework for reusable components

5
Talk Outline

• Motivation
• DHTs and DOLRs
• A Flexible Routing API
• Usage Examples

6
Decomposing Functional Layers

• Distributed Hash Tables (DHT)
• put(key, data), value get(key)
• Hashtable layered across network
• Handles replication distributes replicas
  randomly
• Routes queries towards replicas by name
• Decentralized Object Location and Routing (DOLR)
• publish(objectId), route(msg, nodeId),
  routeObj(msg, objectId, n)
• Application controls replication and placement
• Cache location pointers to replicas queries
  quickly intersect pointers and redirect to nearby
  replica(s)

7
DHT Illustrated
8
DOLR Illustrated
9
Architecture
Tier 2
CFS
PAST
SplitStream
i3
OceanStore
Bayeux
Multicast
DHT
DOLR
CAN, ChordDHash
Tapestry PastryScribe
Tier 1
Replication
Routing Mesh
Tier 0
10
Talk Outline

• Motivation
• DHTs and DOLRs
• A Flexible Routing API
• Usage Examples

11
Flexible API for Routing

• Goal
• Consistent API for leveraging routing mesh
• Flexible enough to build higher abstractions
• Openness promotes new abstractions
• Allow competitive selection to determine right
  abstractions
• Three main components
• Invoking routing functionality
• Accessing namespace mapping properties
• Open, flexible upcall interface

12
API (routing)

• Data types
• Key, nodeId 160 bit integer
• Node Address (IP port ), nodeId
• Msg application-specific msg of arbitrary size
• Invoking routing functionality
• Route(key, msg, node)
• route message to node currently responsible for
  key
• Non-blocking, best effort message may be lost
  or duplicated.
• node transport address of the node last
  associated with key (proposed first hop, optional)

13
API (namespace properties)

• nextHopSet local_lookup(key, num, safe)
• Returns a set of at most num nodes from the local
  routing table that are possible next hops towards
  the key.
• Safe whether choice of nodes is randomly chosen
• nodehandle neighborSet(max_rank)
• Returns unordered set of nodes as neighbors of
  the current node.
• Neighbor of rank i is responsible for keys on
  this node should all neighbors of rank lt i fail
• nodehandle replicaSet(key, num)
• Returns ordered set of up to num nodes on which
  replicas of the object with key key can be
  stored.
• Result is subset of neighborSet plus local node
• boolean range(node, rank, lkey, rkey)
• Returns whether current node would be responsible
  for the range specified by lkey and rkey, should
  the previous rank-1 nodes fail.

14
API (upcalls)
Application
Application

• Deliver(key, msg)
• Delivers an incoming message to the application.
  One application per node. Demultiplexing done by
  including demux key in msg.
• Forward(key, msg, nextHopNode)
• Synchronous upcall invoked at each node along
  route
• On return, will forward msg to nextHopNode
• App may modify key, msg, nextHopNode, or
  terminate by setting nextHopNode to NULL.
• Update(node, boolean joined)
• Upcall invoked to inform app of a change in the
  local nodes neighborSet, either a new node
  joining or an old node leaving.

deliver
forward
msg
msg
msg
Routing Layer
Routing Layer
15
Talk Outline

• Motivation
• DHTs and DOLRs
• A Flexible Routing API
• Usage Examples

16
DHT Implementation

• Interface
• put (key, value)
• value get (key)
• Implementation (source S, root R)
• Put route(key, PUT,value,S, NULL)Reply
  route(NULL, PUT-ACK,key, S)
• Get route(key, GET,S, NULL)Reply route(NULL,
  value,R, S)

17
DOLR Implementation

• Interface
• RouteNode(msg, nodeId)
• Publish(objectId)
• RouteObj(msg, objectId, n)
• Implementation (server S, client C, object O)
• RouteNode route(nodeId, msg, NULL)
• Publish route(objectId, publish,O,S,
  NULL)Upcall addLocal(O,S)
• RouteObj route(nodeId, n,msg, NULL)Upcall
  serverSet getLocal(O)if (serverSetltn),
  route(nodeId, n-serverSet,msg, NULL)for
  first n entries in serverSet,
  route(serverSeti, msg, NULL)

18
Conclusion

• Very much ongoing work
• Feedback valuable and appreciated
• Ongoing Work
• Implementations will support routing API
• Working towards higher level abstractionsDistribu
  ted Hash Table APIDOLR publish/route API
• For more information, see IPTPS 2003
• Thank you

19
Backup Slides Follow
20
Storage API Overview

• linsert(key, value)
• value lget(key)

21
Storage API

• linsert(key, value) store the tuple ltkey, valuegt
  into local storage. If a tuple with key already
  exists, it is replaced. The insertion is atomic
  wrt to failures of the local node.
• value lget(key) retrieves the value associated
  with key from local storage. Returns null if no
  tuple with key exists.

22
To Do

• Following slides contain functions that we
  havent decided on yet

23
Basic DHT API Overview

• insert(key, value, lease)
• value get(key)
• release(key)
• Upcalls
• insertData(key, value, lease)

24
Basic DHT API

• insert(key, value, lease) inserts the tuple
  ltkey, valuegt into the DHT. The tuple is
  guaranteed to be stored in the DHT only for
  lease time. value also includes the type of
  operations to be performed on insertion. Default
  operation types include
• REPLACE replace value associated with the same
  key
• APPEND append value to the existing key
• UPCALL generate an upcall to application before
  inserting

25
Basic DHT API

• value get(key) retrieves the value associated
  with key. Returns null if no tuple with key
  exists in the DHT.

26
Basic DHT API

• ltkey, valuegt scan(searchKey) let n denote the
  node responsible for searchKey. Return the tuple
  stored at n whose key follows searchKey. If no
  such a key exists, returns the first key that
  follows searchKey and which is not mapped on n.
• Notes
• If there is a tuple whose key is equal to
  searchKey, then that tuple is returned
  (degenerates to get() operation)
• This function assumes a total order amongst the
  keys.
• This function can be used to implement global
  scan in DHT

27
Basic DHT API

• Release(key) releases any tuples with key from
  the DHT. After this operations completes, tuples
  with key are no longer guaranteed to exist in the
  DHT.

28
Basic DHT API Open questions

• Semantics?
• Verification/Access control/multiple DHTs?
• Caching?
• Replication?
• Should we have leases? It makes us dependent on
  secure time sync.

29
Replicating DHT API

• Insert(key, value, numReplicas) adds a
  numReplicas argument to insert. Ensures
  resilience of the tuple to up to numReplicas-1
  simultaneous node failures.
• Open questions
• consistency

30
Caching DHT API

• Same as basic DHT API. Implementation uses
  dynamic caching to balance query load.

31
Resilient DHT API

• Same as replicating DHT API. Implementation uses
  dynamic caching to balance query load.

32
Publish API Overview

• Publish(key, object)
• object Lookup(key)
• Remove(key, object)

33
Publish API

• Publish(key, object) ensures that the locally
  stored object can be located using the key.
  Multiple instances of the object may be published
  under the same key from different locations.
• object Lookup(key) locates the nearest
  instance of the object associated with key.
  Returns null if no such object exists.

34
Publish API

• Remove(key, object) after this operation
  completes, the local instance of object can no
  longer be located using key.