Franck Petit

1
Stabilization, Synchronization of Logical Clocks
and Applications

• Franck Petit
• INRIA / LIPLyon

2
Preliminaries

• G(V,E) is a connected network
• V set of n nodes/processes
• E set of m bidirectional links
• Np set of neighboring nodes of p
• Memory shared between neighboring nodes

3
Preliminaries, Distributed Algorithm

• In each computation step
• According to its variables and the variables of
  its neighbors, a node/process is either enabled
  to execute an action or not
• Synchronous system
• Every enabled nodes execute an action atomically
• Asynchronous system
• Some enabled nodes are chosen by an unfair
  adversary
• The chosen nodes execute an action atomically

4
Logical Clock Synchronization

• Each node p maintains a logical clock register rp

4 3 2 1 0

• Synchronous/Asynchronous Environment

rp rp 1
5
Logical Clock Synchronization

• Each node p maintains a logical clock register rp

4 3 2 1 0

• Synchronous/Asynchronous Environment

rp rp 1
Synchronizer
6
Global Synchronizer (asynchronous) distributed
systems
In systems with unique IDs or a particular node
(root, leader, main server)?

• Wave Algorithms, e.g.,
• Propagation of Information with Feedback (PIF)
• Depth-First Token Circulation
• Global Synchronization, e.g.,
• (Group) Mutual Exclusion
• Leader Election
• Reset
• Logical Clock Synchronization
• Rooted Spanning Tree

7
Propagation of Information with Feedback
r
8
Propagation of Information with Feedback
r
9
Propagation of Information with Feedback
r
10
Propagation of Information with Feedback
r
11
Propagation of Information with Feedback
r
12
Propagation of Information with Feedback
r
13
Propagation of Information with Feedback
r
14
Propagation of Information with Feedback
r
15
Propagation of Information with Feedback
r
16
Propagation of Information with Feedback
r
17
Propagation of Information with Feedback
r
18
Propagation of Information with Feedback
r
19
Propagation of Information with Feedback
r

• For One Puse
• Best Case O(D)
• Worst Case O(N)

Can we provide better complexities?
20
Logical Clock Synchronization

• Each node p maintains a phase clock register rp

4 3 2 1 0

• Synchronous/Asynchronous Environment

If ?q ? Np rp rq then rp rp 1
21
Logical Clock Synchronization
Unison

• Each node maintains a phase clock registerr in
  0,,K-1ZK

• Safety The gap between the phase clock of two
  neighboring nodes is at most equal to 1 (mod K).
• No starvation (Vivacity) Each phase clock r is
  incremented by 1 (mod K) infinitely often.

22
Logical Clock Synchronization
Unison
4 3 2 1 0
Minimality of K ?
K5
23
Logical Clock Synchronization

• K2?
• No possible order among the clocks

Synchronous
1 0
24
Logical Clock Synchronization

• K2?
• No possible order among the clocks

Asynchronous
1 0
25
Logical Clock Synchronization

• Successor Function and Predecessor Function
  possible if K3
• K3
• Local total order l over Z30,1,2

a l b iff 0 b-a mod 3 1
0 l 1 1 l2 2 l 0
26
Logical Clock Synchronization

• K3, Lifting

27
Logical Clock Synchronization

• K3, Lifting

0
1
1
0
1
D0
1
? 0? ? 1
0
1
0
1
1
D0
? 0? ? 1
1
28
Logical Clock Synchronization

• K3, Lifting

29
Logical Clock Synchronization

• K3, Lifting

30
Logical Clock Synchronization

• K3, Lifting

31
Logical Clock Synchronization
The lifting is compatible with the local
ordering of ZK
q.r
Lifted execution
q.R
p.r
D0
D1
D2
? 0? ? 1 ? ? 2?
p.R
p.R q.R
D0
D1
D2
? 0? ? 1 ? ? 2?
p.r l q.r
Execution
The lifting defines a global preorder over the
nodes
32
Generalization over ZK

• (Local) total order l over ZK0,1,2,,K
• Let M 1 and K 2M1

a l b iff 0 b-a mod K M
With M2 and K5, then for 1 4 l 1 0 l1 1
l 1 1 l 2 1 l 3
33
Complexities

• Space O(1)
• Time, for one pulse
• Best Case O(1)
• Worst Case O(D) ? DELAY

34
Fault Tolerance
Starting from an arbitrary configuration ?
35
Self-stabilization
A self-stabilizing system, regardless of its
initial state, is guaranteed to converge to the
intended behavior in finite time. Dijkstra 74

• Transient Faults

36
After a fault or an arbitrary initialization

• K3

D0
D1
D2
? 0? ? 1 ? ? 2?
D0
D1
D2
? 0? ? 1 ? ? 2?
37
After a fault or an arbitrary initialization

• K3

0
1
2
0
D0
D1
D2
1
? 0? ? 1 ? ? 2?
5
2
4
2
1
D0
D1
D2
3
? 0? ? 1 ? ? 2?
0
DEADLOCK!
38
After a fault or an arbitrary initialization

• K6

39
After a fault or an arbitrary initialization

• K6

40
After a fault or an arbitrary initialization

• K6

MUTUAL EXCLUSION!
41
After a fault or an arbitrary initialization
What are the conditions to be able to
(re)-synchronize the system ?
42
System Configurations

• ? The register values are arbitrary

?1 The register of two neighboring process is
less than or equal to 1
?01 There is no deadlock (there exists a
compatible lifting to the configuration)
K4
43
System Configurations
? The register values are arbitrary
?1 The register of two neighboring process is
less than or equal to 1
?01 There is no deadlock (there exists a
compatible lifting to the configuration)
THEOREM KgtCG-gt ?1 ?10

• CG (CG ? n), the cyclomatic characteristic of the
  graph Equal to the size of the greatest cycle
  in one of the cycle basis of G where the size of
  the greatest cycle is minimum (equal to 2 if G is
  acyclic)

Boulinier, Petit, and Villain, PODC 2004
44
Cyclomatic characteristic of G
CG4 in meches and tories
45
Cyclomatic characteristic of G
CG6 in honeycombs
46
Cyclomatic characteristic of G
CG4 in hypercubes
47
Cyclomatic characteristic of G
CGn on rings
48
Logical Clock Synchronization

• To avoid deadlock due arbitrary initial values, K
  must be greater than CG (CG ? n), the cyclomatic
  characteristic of the graph

49
System Configurations
? The register values are arbitrary
?1 The register of two neighboring process is
less than or equal to 1
?01 There is no deadlock (there exists a
compatible lifting to the configuration)
?
?
?10
?1
No Deadlock
50
Stabilization

• Global Reset (Stabilizing PIF), implies a root or
  IDs
• Local Reset (also woks in anonymous networks)

QUESTION What is the motivation behind
anonymity?
Only a few amount of bits allows to distinguish
a huge number of nodes!
51
Advantages of Anonymous Solutions

• Lack of (underlying) infrastructure
• No unique identifier assignation or no central
  process
• No maintenance of any distributed structure
• Economic advantages
• User privacy preserved Delporte-Gallet,
  Fauconnier, Guerraoui, and Ruppert, OPODIS 2007
• No one-to-one routing

52
Self-Stabilizing Logical Clock Synchronization

• To avoid starvations due arbitrary initial
  values, K must be greater than CG (CG ? n), the
  cyclomatic characteristic of the graph

• Safety eventually guaranteed using a reset
  mechanism Register r is set in -?,,0
• To avoid starvations due arbitrary initial
  values, ? must be greater than than HG1 (HG ?
  n), the greatest chordless cycle of the graph
  Boulinier, Petit, and Villain, PODC 2004

53
Self-Stabilizing Logical Clock Synchronization
SSSync AN ?q?Np Correctp(q) ?ltltapplicationgtgtrp
?(rp) AR ? (?q?Np Correctp(q) )? (rp?tail?)
? rp-? //reset AC rp? init??( ?q?Np
rq?init??rp?(rq) ? rp?(rp)
54
Asynchronous, Anonymous Logical Clock, Related
Works
of states Stabilizing Time
O(n²)
O(nd)
Gouda, Couvreur, Francez, 1992
Dolev, 2000
O(n²)
O(d)

• SSSync(K,a,M)

O(nd)
a0, KgtM.CG, MgtHG-2
O(nd)
O(n)
O(n)
agtHG-2, M1, KgtCG Sur un arbre
O(d)
O(1)
55
Wave Algorithms in Anonymous Networks

• All existing solutions for coordination problems
  (either global, local or at distance ?) are based
  on
• Existence of unique IDs (or a root)
• Underlying wave mechanism

QUESTION Can we provide wave algorithms using
neither IDS nor root, i.e., working in an
anonymous system?

• Anonymous global (? Diam) waves for non
  fault-tolerant systems Tel, Concurrency88

56
Logical Clock Synchronization in Asynchronous
Settings

• Let us observe a possible execution of the
  self-stabilizing phase clock synchronization
  again
• The system (a chain of 4 nodes) is stabilized
• K5

0
1
2
3
4
0
0
1
2
3
4
0
2
1
3
4
0
1
2
3
4
0
P0
P1
P2
P3
P4
time
During the execution, coherent phase cuts Pi are
built defined by the same value i on each clock
register.
57
Logical Clock Synchronization in Asynchronous
Settings

• Let us observe a possible execution of the
  self-stabilizing phase clock synchronization
  again
• The system (a chain of 4 nodes) is stabilized
• K5

0
1
2
3
4
0
0
1
2
3
4
0
2
1
3
4
0
1
2
3
4
0
P0
P1
P2
P3
P4
time
During the execution, coherent phase cuts Pi are
built defined by the same value i on each clock
register.
58
Phase Clock Synchronization in Asynchronous
Settings

• Observe that
• After stabilization, the phase clock
  synchronization algorithm provides a wave stream

Stabilization(Convergence)
0
1
2
D
D1
D2
0
1
2
D
D1
D2
0
1
2
D
D1
D2
0
1
2
D
D1
D2

• In the future of a coherent phase Pi , the dth
  phase clock incrementation causally depends on
  nodes at distance d.

59
Self-Stabilizing ?-Wavelets

• When the clock register r is equal to a
  particular value, e.g., 0, add a particular event
  DECIDE

THEOREM The self-stabilizing phase clock
synchronization provides a self-stabilizing
?-wavelet mechanism
60
Self-Stabilizing ?-Wavelets

• For instance, if ? Diam, the above protocol
  provides a Propagation of Information with
  Feedback over an anonymous network

Propagation
Feedback
0
D
2D
0
D
2D
0
D
2D
0
D
2D
61
Self-Stabilizing ?-Local Infimum Computation ?

• An infimum ? over a set S is an associative,
  commutative, and idempotent (x?xx) binary
  operator.

• For instance, ? ?

Propagation
Feedback
a,b,c,f
0
D
2D
b
a,b,c,d,e,f
0
D
2D
a,e
0
D
2D
d
0
D
2D
62
Self-Stabilizing ?-Local Infimum Computation ?

• An infimum ? over a set S is an associative,
  commutative, and idempotent (x?xx) binary
  operator.

• For instance, ? min

Propagation
Feedback
6
0
D
2D
2
0
0
D
2D
9
0
D
2D
0
0
D
2D
63
Self-Stabilizing ?-Barrier Synchronization

• A two-layer phase clock algorithm
• The upper layer clock is driven by the lower
  layer clock
• In other words, every node moves the upper layer
  clock rUP from i to i1 (mod KUP) each time the
  lower layer clock incremented the lower layer
  clock, rLOW, ? times
• The upper layer implements a self-stabilizing ?
  -Barrier Synchronization, i.e., Phase i1 (mod
  KUP) is executed by each node only after all
  nodes at distance less than or equal to ? have
  completed Phase i (mod KUP)

64
Self-Stabilizing ?-Local Resource Allocation

• Let ? be a reflexive binary compatibility
  relationship over A, a set of resources.
• If (a,b) ? ?, then a and b are compatible,
  otherwise a and b are conflicting.
• The ?-Local Resource Allocation is specified as
  follows
• Safety If two nodes at distance less than or
  equal to ? execute their critical section
  simultaneously, then both are using compatible
  resources
• No starvation (Vivacity) If a node requests a
  resource to enter the critical section, then its
  request is eventually satisfied and it enters the
  critical section

65
An example ?-Local Mutual Exclusion

• Let A be a set of IDs s.t. for every pair (a,b)
  in A, a (resp. b) is conflicting with b (resp.a)
  (i.e., ??)
• Given two neighboring nodes P,Q, rP (resp. rQ)
  being the upper clock of P (resp. Q)
• (rP ,IDP) ? (rQ ,IDQ)iff
• (rP lt(mod K) rQ) or ((rP (mod K) rQ) and
  (IDPltIDQ))
• Mutual Exclusion at distance ? is provided by
  applying the order relation ? in each phase
  transition
• The stabilization time is in O(n) time units
• The space complexity is in O(log Diam)

66
Enabling Snap-Stabilization in Anonymous Networks
QUESTION Can we design snap-stabilizing waves
algorithms in anonymous networks?
67
Self-stabilization
A self-stabilizing system, regardless of its
initial state, is guaranteed to converge to the
intended behavior in finite time. Dijkstra 74
68
Stabilization and Safety
A priori, Safety is not guaranted during the
faults
A self-stabilizing system, regardless of the
initial state of the processors, is guaranteed to
converge to the intended behavior in finite
time. Dijkstra 1974
Safety is lost during faults and system
stabilization.
A snap-stabilizing system, regardless of the
initial state of the processors, is guaranteed to
converge to the intended behavior in 0
times. Bui, Datta, Petit, Villain 1999
Safety is lost during faults only.
69
Users Point of View
Network
X
F(X)
F(.)
70
Users Point of View
F(.)
71
Users Point of View
F(.)
Self-Stabilizing Systems
72
Users Point of View
F(.)
Snap-Stabilizing Systems
73
Self-Stabilization
stabilization phase
correct
illegitimate
legitimate
A general technique to deal with transient faults
74
Self-Stabilization
Request
Request
Request
Request
stabilization phase
correct
correct
illegitimate
Transient Fault
legitimate
stabilization time
enabled starting actions
legitimate initial configuration
75
Snap-Stabilization
Request
correct
always correct
abnormal
Transient Fault
normal
first enabled starting action
Quality of Service
76
Coherent Cuts Detection
If I execute 3 consecutive normal actions, my
neighbors execute at least one
0
1
2
1
0
2
77
Coherent Cuts Detection
Generalization
If I execute x consecutive normal actions, then
my neighbors execute at least x-2
0
1
2
n
1
0
n
n-1
2
78
Coherent Cuts Detection
I can extend the phenomenon to my neighbors
0
1
n
n-1
1
n-1
2
n-2
3
n-3
79
Coherent Cuts Detection
After an enought long normal action sequence
(NA), a coherent phase is guaranteed in the past.
Cu, CuD
10
11
12
20
13
14
15
16
17
18
19
11
19
15
16
14
12
18
15
16
14
13
17
15
16
14
14
15
16
80
Coherent Cuts Detection
0 can mask a reset. If I want to know the
behavior of the network, I must avoid this value
2
1
1
0
-a

0
1
Reset
81
Snap-Stabilization Wave Protocol for Anonynous
Networks

• After an enought long normal action sequence
  (NA), a coherent phase is guaranteed in the past.
• A wave can computed after a sequence of 2D3
  normal actions passing through 0 and with an
  initial value greater than D.

82
Snap-Stabilization Wave Protocol for Anonynous
Networks
A wave is requested by an application
K gt 3D5
D lt clock lt K-2D-4
No
Yes
2D3 consecutive incrementations
DECIDE
No, a reset occured
Yes
83
Enabling Snap-Stabilization in Anonymous Networks

Application layer
Request
Answer
Snap-Wave Toolbox
Underlayer Unison
84
Some references
http//graal.ens-lyon.fr/fpetit

• UNISON and its applications
• Boulinier, Petit, Villain PODC 04When Graph
  Theory helps Self-Stabilization
• Boulinier, Petit, Villain, Algorithmica
  08Synchronous vs. Asynchronous Unison
• Boulinier, Petit, APDCM 08 Self-Stabilizing
  Wavelets and ?-Hop Coordination
• Boulinier, Levert, Petit ICDCN
  08Snap-Stabilizing Waves in Anonymous Networks
• Snap-Stabilization
• Bui, Datta, Petit, Villain Distributed
  Computing 07Snap-stabilization and PIF in Tree
  Networks