Chord

1

• Chord CFS

Presenter Gang Zhou Nov. 11th, 2002 Email
gz5d_at_cs.virginia.edu University of Virginia
2
Relation of Chord, CAN, INS, CFS DCS
Polynomial in number of attributes in name
specifiers
3
Outline of Chord

• Chord protocol
• What is Chord ?
• Why use Chord ?
• Basic Chord protocol
• Extended Chord protocol
• Simulation results
• Chord system
• APIs provided by Chord system

4
Outline of CFS

• CFS system structure
• Chord layer
• Server selection
• DHash layer
• Replication
• Caching
• Load balance
• Quota
• Update and delete
• Experimental results

5

• Part One Chord

6
What is Chord?
node n that stores keys value
Application in n
node n
7
Why use Chord ?

• Chord scales well
• each node maintains information only about
  O(logN) other nodes

• High efficiency
• resolves a lookup via O(logN) messages to other
  nodes

• Fault tolerance

8
Base Chord protocol

• What is identifier cycle?
• Where to store a key?
• How to look up a key quickly ?
• Node joins (departures)

9
The base Chord protocol

• What is identifier cycle ?

10
Where is a key stored?

• Successor (k) the first node encountered when
  moving in the clockwise direction starting from k
  in the identifier circle.

11
How to look up a key quickly ?
n1
Key
move
12
How to look up a key quickly ?(cont.)

• We need finger table

13
How to look up a key quickly ?(cont.)

• finger table for node 1

14
How to look up a key quickly ?(cont.)
Predecessor(1) ? successor(1)
Value of key 1 ?
15
Node joins

• Two challenges
• Each nodes finger table is correctly filled
• Each key k is stored at node successor(k)

• Three operations
• Initialize the predecessor and fingers of the new
  node n
• Update the fingers and predecessors of existing
  nodes
• Copy to n all keys for which node n has became
  their successor

16
Initialize the predecessor and fingers of node n

• Idea Ask an existing node for information needed

Join in
17
Update the fingers and predecessors of existing
nodes

• Observation when node n joins the network, n
  will become the ith finger of a node p when the
  following two conditions meet
• P proceeds n by at least 2i-1
• The ith finger of node p succeeds n

• Solution Try to find predecessor(n- 2i-1) for
  all 1ltiltm and check whether n is their ith
  finger, and whether n is their predecessors ith
  finger.

18
Update the fingers and predecessors of existing
nodes (cont.)
Predecessor(6-21-1) 3, update
Join in
?6
?6
?6
?6
19
Copy to n all keys for which node n has became
their successor

• Idea Node n can become the successor only for
  keys stored by the node immediately following n

Join in
20
Extended Chord protocol

• Concurrent joins
• Failures and replication

21
Concurrent joins

• When multiple nodes with similar identifiers join
  at the same time,
• They tell the same predecessor that they are its
  successor.
• How to update predecessor and successor ?

22
Concurrent joins (cont.)

• Example Node b joins

a
d
23
Failures and replication

• Challenges
• When node n fails, successor (n) must be found ?
• Successor (n) must has a copy of the key/value
  pair
• ns failure must not disrupt queries in progress
  as the system is re-stabilizing

24
How to find ns successor after n fails ?

• To find
• ns predecessor (n2) found that n doesnt respond
  in stabilizing
• To recover
• n2 looks through n2s finger table for the first
  live node n1
• n2 asks n1 for successor(n2) and uses the result
  as n2s new successor

25
How to ensure that ns successor has a copy after
n fails ?

• Each Chord node maintains a list of its r nearest
  successors. Each successor has the data copy.
• After node n fails, queries for its keys
  automatically end up at its successor

26
How to keep in-progress queries undisrupted ?

• Detect failure
• Before stabilization has completed, node failure
  can be detected by timing out the requests
• Continue query
• Any node with identifier close to the failed
  nodes identifier will have similar table entries
• Such node can be used to route requests at a
  slight extra cost in route length

27
Simulation results

• Path length the number of nodes traversed by a
  lookup operation

Average O(logN)
28
Simulation results (cont.)
lt
Miss rate due to state inconsistency
miss rate due to node failures (key lost)

• Miss rate due to state inconsistency increases
  fast with failure frequency

29
APIs provided by Chord system
30

• Part Two CFS

31
CFS System structure
32
Chord Layer --- Server selection
n

• Factors to consider
• Distance around the ID ring
• RPC latency

key
n3
n2
n1
33
DHash layer --- Replication

• DHash places a blocks replicas at the k servers
  immediately after successor (block).
• After successor (block) fails, the block is
  immediately available at the new successor
  (block)
• Independence failure is provided
• close to each other in the ID ring ? physically
  close to each other

34
DHash layer --- Caching

• How to cache?
• Cache replacement? --- LRU
• Cache vs. replication?

• Replication is good for solving nodes failure
• Cache is good for loading balance

35
DHash layer --- Load balance

• Break file systems into many distributed blocks
• Caching
• A real server can act as multiple virtual servers
  
• --- ID is derived from hashing both the real
  servers IP address and the index of the virtual
  server

36
DHash layer --- Quota

• Why quota?
• The total amount of storage an IP address can
  consume will grow linearly with the total number
  of CFS servers
• Prevent malicious injection of large quantities
  of data

Example If each CFS server limits any one IP
address to using 0.1 of its storage, then an
attacker would have to mount an attack from about
1000 machines for it to be successful
37
DHash layer --- Update and delete

• Update
• Content hash block supplied key
  SHA-1(blocks content)
• Root block only publisher with private key can
  change it
• Delete
• No delete, useful for recovering from malicious
  data insertion
• (Good or not?)

38
Experimental results

• lookup cost O(logN)

39
Experimental results (cont.)

• caching

1000 servers
40
Experimental results (cont.)

• Effect of nodes failure --- lookup fail because
  of all replicas fail

• 6 replicas
• 1000 blocks
• 1000 servers

41
Some discussion

• Chord in sensor network ?
• Do you want to use CFS (since no delete) ?
• Build CFS over CAN and INS ?
• Lazy replica copying ?