Routing

1
Routing Naming

• Routing naming in distributed
• environments
• 1. lecture Chord
• 2. lecture Tapestry
• Questions.

2
Chord Introduction

• A lookup service
• Scalable.
• Available.
• Fully distributed.
• Consistent hashing for key node id assignment.
• (Load balancing).

3
Chord How is it used?

• The Chord software is a library to be linked with
  the application that uses it (client/server).
• lookup(key) return the ip-address of the node
  responsible for key, where responsible means the
  resulting node from the key-to-node mapping done
  by Chord.
• Informs application software about joining nodes,
  and can inform when nodes fail.
• Everything else must be done by the application,
  this includes authentication, caching,
  replication, user-friendly naming of data and
  moving data/pointers to data.

4
Chord Overview

• With high probability
• Load balancing (because of consistent hashing).
• Each node needs only to know about O(log N) other
  nodes.
• Lookup requires O(log N) messages.
• Joins leaves requires only O(log2 N) messages.
• Joins leaves only maps O(1/N) fraction of the
  keys to a new node.

5
Chord Basics

• Consistent hashing assigns each node key an
  m-bit identifier using for instance SHA-1.
• m must be large enough to insure a negligible
  probability of two nodes or keys being assigned
  the same identifier.
• Keys maps to the node successor(key).

6
Chord Basics(2)

• Identifier circle (modulo 2m)

7
Chord Key location

• Naive key location (successor pointers) requires
  O(N) messsages.
• Finger table on each node
• Contains at most m entries.
• Ith entry contains
• Successor(n2i-1 modulo 2m), 1 i m.
• Validity interval.
• Start id.
• The ith entry is called the ith finger.

8
Chord Finger tables

• Characteristics
• Small amount of data stored.
• Each node knows more about nodes closely
  following it on the identifier circle.
• Not enough data in a nodes fingertable to find
  all keys without contacting other nodes.

9
Chord Key location(2)

• Locating keys using finger tables
• If successor(key) is present in finger table, we
  are done.
• If not, contact the last node in the finger
  table, that lies before the key. This node will
  then do the same until a node has been reached,
  that knows successor(key).
• With high probability this requires O(log N)
  messages.
• Intuitively Search space is cut in half each
  iteration.

10
Chord Key location(3)

• An example

11
Chord Node joins

• To do list
• Initialize finger table predecessor pointer.
• Update finger tables, predecessor successor
  pointers of existing nodes.
• Notify application of the new node, so keys can
  be moved.
• Must know one existing node (n).

12
Chord Node joins(2)

• Initializing finger table
• The new node (n) asks n to lookup the successor
  of n. This becomes ns successor its
  predecessor becomes ns predecessor.
• Then n asks n to lookup the keys in ns finger
  table, which requires O(m log N) messages.
• This can be sped up by checking whether the ith
  finger is in the interval of the i1th finger.
  If not the ith finger is also the i1th finger.
  
• This reduces required messages to O(log2 N).

13
Chord Node joins(3)

• Updating finger tables pointers of existing
• nodes
• Runs through the identifier circle in reverse and
  updates the fingertables if n is between
  (n2i-1) and the finger.
• n can calculate which nodes at which levels it
  might need to update.
• Insert itself as the predecessor of its
  successor.
• Transfering keys
• Done by the application, not Chord.

14
Chord Node joins(4)

• An example

Before and after node 6 joins.
15
Chord Concurrency

• Joins only set successor predecessor pointer on
  the joining node initializes/fixes finger table
  at a later stage.
• Stabilize runs periodically
• the node asks its successor for its predecessor.
• If a new successor has been found it will be
  notified of the change.
• Fix fingers also runs periodically
• The node picks a random finger and validates it
  through successor pointers.
• Successor-lists are maintained in case of
  successor failure.

16
Chord Concurrency(2)

• Lookups can fail if the network has not been
  stabilized. This is not handled by Chord, but it
  is up to the application to deal with this.
• Lookups may still work although slower if the
  finger tables are not correct.
• With a successor list length of log N there is a
  high probability of successor(key) actually
  finding the successor with O(log N) messages if
  the probability of a node failing is 50.

17
Chord Simulation

• Uses iterative style lookup, not recursive.
• Load balancing...
• With an average of 50 keys per node during
  simulation it was possible to have zero keys on
  some nodes and up to 457 keys per node.
• With the introduction of virtual nodes this is
  improved substantially at the cost of extra data
  being stored extra network traffic.

18
Chord Experiment

• Scalability is pretty good.

60 ms roundtrip. Only 10 real nodes though.
19
Chord Summary

• A fully distributed lookup service.
• Scalable.
• Load balancing is not optimal meaning that the
  bounds on messages in connection with lookups,
  joins etc. are not valid. Virtual nodes can fix
  this with certain costs.
• Does not exploit locality.

20
Tapestry Introduction

• A lookup routing service.
• Fully distributed.
• Fault tolerant.
• Scalable.
• Hotspot handling.
• Exploits locality.

21
Tapestry Plaxton

• Assumes hashing of keys nodes using e.g. SHA-1
  for load balancing.
• Uses neighbour maps for routing.
• Neigbour map
• for 0642

22
Tapestry Plaxton(2)

• Routes digit by digit from the right, this is
  arbitrary.
• Each hop cuts a digit from the search space, this
  means that the number of hops is bound by O(logb
  N).

23
Tapestry Plaxton(3)

• An example of routing from node 0325 to
• node 4598

24
Tapestry Plaxton location

• Each key is assigned a root node responsible for
  knowing the location of the node having the
  object associated with the key.
• When a node publishes an object it sends a
  message to the root node. Every node en route to
  the root node stores the location information.
• Key-root node assignment requires global
  knowledge.

25
Tapestry Plaxton pros cons

• Pros
• Simple fault handling.
• Scalable.
• Exploiting locality.
• Proportional route distance.
• Cons
• Global knowledge required.
• Root node vulnerability.
• No hotspot handling.

26
Tapestry A Plaxton extension

• Inspired by Plaxton.
• Uses backpointers to keep track of nodes having
  itself in their neighbour maps.
• Stores all known locations of the same object.
• Also stores metrics about objects.

27
Tapestry Fault handling

• Uses soft state (a sort of leasing).
• Fault tolerant routing.
• Uses keep-alive messages.
• Keeps backup neighbours in neighbour map.
• Marks failed node in neighbour map as invalid
  routes a small percentage of messages to it for
  some time before promoting a backup neighbour,
  because insertions are expensive.

28
Tapestry Fault handling(2)

• Fault tolerant location.
• Uses multiple roots, selected by adding a global
  constant sequence of numbers to the ID.
• Object servers must republish location
  information because of soft state.

29
Tapestry Surrogate routing

• Global knowledge not possible.
• The root is chosen by attempting to route to the
  node having the same id as the key. If this node
  exist all is well. If not at some time an empty
  neighbour map will be encountered. Then an
  alternative link will be deterministically
  chosen. This continues until there are no more
  digits or a node with a neighbour map containing
  only itself is reached.

30
Tapestry Dynamic node joins

• A new node N must know a gateway Tapestry node G.
• To populate its neighbour map N attempts to route
  to its own id using G. At each hop en route N
  copies the present nodes neighbour map.
• N optimizes its neighbour map by iteratively
  requesting its neighbours neighbour maps until
  only small gains are possible.

31
Tapestry Dynamic node joins(2)

• Node N must also notify its neighbours of its
  presence, so that they can update their
  backpointers.
• Surrogate roots are also notified of Ns presence
  to maintain the surrogate routing properties.

32
Tapestry Explicit updates

• Tradeoff between republishing intervals
  constistency of location pointers.
• Pro active explicit node leaves.
• The leaving node N notifies all servers having
  objects, for which N is root server, causing them
  to republish their objects.
• Invalid location references are detected. I.e. If
  a query is forwarded to an old server it can
  notify the routing node, which then deletes the
  reference.

33
Tapestry Mobile objects

• An object moving from server A to B
• A points to B.
• B republishes the object. When this message
  reaches the junction where it intersects the old
  publishing path a message is sent towards A. When
  the original message reaches the root node it
  sends a message to A, causing the pointer to B to
  be deleted.

34
Tapestry Mobile objects(2)

• An example

35
Tapestry Optimizations

• Pings are used to measure network latency. A rise
  in this causes the node to search for another
  neighbour node.
• Hotspot caching is used for both object and
  references

36
Tapestry Performance

• Location pointers

RDP Relative Distance Penalty.
37
Tapestry Performance(2)

• Multiple roots running on 4096 nodes in a 5000
  node artificially
• generated topology

38
Tapestry Performance(3)

• Simple Tapestry vs. Directory server simulation

39
Tapestry Summary

• Fault tolerant.
• Exploits locality as opposed to Chord.
• Slow node insertions.
• Lookups/routing is bound by O(logb N) hops.
• Supports mobile objects.
• Graceful performance degredation under load.

40
Questions

• How could the successor-list in Chord be
  maintained?
• How could authentication replication by
  implemented by an application using Chord?
• What happens in Tapestry when there are not
  enough nodes to fill the neighbour map?
• Describe how a root node for an object in
  Tapestry is chosen.
• Does Chord and Tapestry exploit locality? If so
  how? If not how could it be implemented?