Distributed Algorithms Models of Distributed Systems

1
Distributed AlgorithmsModels of Distributed
Systems

• Ali Ghodsi Royal Institute of Technology
• aligh AT kth.se

2
Models

• What is a model?
• An abstraction of the relevant properties of a
  system
• Why construct or learn a model?
• Real world is complex, a model makes assumptions
  and simplifies
• Helps us tackle the complexities
• Often a symbolic model with properties expressed
  in mathematical symbols and relationships

3
Modeling

• What can modeling do for us?
• Useful when solving problems (e.g. making an
  algorithm)
• When predicting behavior (e.g. cost in number of
  messages)
• When evaluating and verifying a solution (e.g.
  simulation)
• Very important skill

4
Modeling

• Different types of models
• Continuous models
• Often described by differential equations
  involving variables which can take real
  (continuous) values
• Discrete event models
• Often described by state transition systems
  system evolves, moving from one state to another
  at discrete time steps
• This course a model of distributed computing
  (discrete)

5
Model of distributed computing

• Biggest challenge when modeling is choosing the
  right level of abstraction!
• The model should be powerful enough to construct
  impossibility proofs
• A statement about all possible algorithms in a
  system
• Our model should therefore be
• Precise explain all relevant properties
• Concise explain a class of distributed systems
  compactly

6
Crash Slide on (Naïve) Set Theory

• Sets contain elements, which can be sets
• Aa, d, a (Set A contains a, and the sets
  d and a)
• Sometimes we do not want to list all the members
  of a set, then we can write
• x x has some property, e.g. x x is an
  even number, the set of all even numbers
• Order not important, number of occurrences not
  important
• a, b a, a, b b, a
• a, a ? a
• Exception multi-sets, number of occurrences are
  important
• a, b ? a, a, b
• If an element x belongs to a set S, we write x?S
• Every sets contains the empty set, ?. I.e. for
  all sets, the following is true ??S

7
Operations on sets, orders

• Cartesian product ?
• X?Y, the set ab a? X and b?Y
• Example, Xa, b, c Yc, d, e
• X?Y ac, ad, ae, bc, bd, be, cc, cd, ce
• Union ??
• X?Y is the set a a? X or a?Y
• Example Xa, b Ya, d, e
• X ? Y a, b, d, e
• Sets are ordered by the subset, ??, relationship
• X?Y iff a?X then a?Y for all a?X
• a, b ? a, b, c
• a, b, c ? a, b, c
• a, d?? a, b, c

8
Informal model of a distributed system

• A distributed system consists of
• a bunch of processors
• connected by a network
• which communicate by message passing
• How do we make a useful abstract model of this?

9
Formal model

• We formalize the whole distributed system as a
  state transition system (STS),
• Commonly used to model discrete systems
• We also formalize each process/node as a STS
• They are commonly used to describe systems,
  algorithms, and software

10
State Transition System (informal)

• A state transition system consists of
• a bunch of states
• rules describing for each state what other states
  it can go to (transition relation)
• a subset of the states which the system can start
  in (initial states)

11
State transition system - example

• Example algorithm Using graphs
• X0
• while (Xlt3) do
• X X 1
• endwhile
• X1
• Formally
• States X0, X1, X2
• Possible transitions X0?X1, X1?X2, X2?X1
• Start states X0

X1
start
X0
X2
12
State transition system - formally

• A STS is formally described as the triple
• (C, ? , I)
• Such that
• C is a set of states
• ? is a subset of C ? C, describing the possible
  transitions (? ? C ? C)
• I is a subset of C describing the initial states
  (I ? C)
• Note that the system may have several transitions
  from one state E.g. ? X2?X1, X2?X0

13
Terminology

• We will use an STS to model 2 different things
• The state of the whole distributed system
• The computation on each local processor
• Two make it clear, we have different names for
  the different parts of the two STSs
• For the STS of the whole distributed system
• The states are called configurations
• The state transitions are called configuration
  transitions
• For the STS of every local process
• The states are called states
• The state transitions are called events

14
Distributed System as a STS

• We will model the whole distributed system as a
  STS (C, ? , I)
• The state of the whole distributed system, C, can
  be described by
• the current configuration of each processor in
  the system
• the messages in transit in the network

15
Executions in the model

• A configuration ? is terminal if there exist no
  transition from ? to any other configuration ??
• An execution in the distributed system is a
  sequence of configurations (?1, ?2, ?3, ) such
  that
• ?1 is an initial configuration, i.e. ?1?I?
• there is a transition between ?i??i1 for every
  i1
• the size of the sequence is either infinite or
  the last configuration is terminal
• A configuration ? is reachable from
  configuration?? if there exists a sequence
• ?? ?1, ?2,,?k?, such that ?i??i1 for all
  1?i?k
• A configuration ? is reachable if it is reachable
  from an initial configuration??

16
Transitions

• What types of transitions are allowed? (?)
• Answer depends on the type of distributed system
• We will model two different types of distributed
  systems, those using
• Synchronous message passing
• Asynchronous message passing

17
Types of message passing

• Synchronous message passing
• The sending and receiving of a message m, happens
  during one configuration transition
• Asynchronous message passing
• The sending of a message m, and the receipt of m
  occur during two different configuration
  transitions (not necessarily consecutively)
• Asynchronous message passing is more general
• Synchronous message passing is a special case of
  asynchronous message passing

18
Transitions

• What exactly happens during a transition,
• Answer again, depends on the type of distributed
  system
• Lets back off a moment, and look at how
  individual processes compute!

19
Asynchronous Message Passing

• Each processor in the distributed system runs a
  local algorithm, which either
• Sends a message
• Receives a message
• Updates its local variables (state)

20
Local Algorithms

• The local algorithm will also be modeled as a
  STS, and in each configuration transition one of
  the following three events happen
• A processor changes state from one state to a
  another state (internal event)
• A processor changes state from one state to a
  another state, and sends a message to the network
  destined to another processor (send event)
• A processor receives a message destined to it and
  changes state from one state to another (receive
  event)

21
Local Algorithms as STS

• Let M be the set of all possible messages that
  can be sent (each one destined to a particular
  process)
• A local algorithm on a processor is modeled by
  the STS ( Z, I, I,S,R ) where
• Z is a set of all the states of the processor
• I is a subset of Z, containing the initial states
  of the STS
• R is a subset of Z?M?Z, describing the possible
  receive events
• S is a subset of Z?M?Z, describing the possible
  send events
• I is a subset of Z?Z, describing the possible
  internal events

22
Example of a local algorithm

• Lets do a local algorithm running on processor p,
  which can receive a ping message, sends back a
  pong and increase a local counter (up to 2
  messages)

23
Ping Pong local algorithm

• M ping, pong
• Z c0, c0r, c0s, c1, c1r, c1s, c2
• I c0
• R (c0,ping,c0r), (c1,ping,c1r)
• S (c0r,pong,c0s), (c1r,pong,c1s)
• I (c0s,c1), (c1s,c2)

24
Ping Pong

• Ping Pong STS
• M ping, pong
• Z c0, c0r, c0s, c1, c1r, c1s, c2
• I c0
• R (c0,ping,c0r), (c1,ping,c1r)
• S (c0r,pong,c0s), (c1r,pong,c1s)
• I (c0s,c1), (c1s,c2)
• Graphically we have the following

c0r
c1r
start
ping
ping
c1
c2
c0
pong
pong
c0s
c1s
25
Model of the distributed system

• Lets go back to the STS of our distributed system
• Based on the STS of a local algorithm, we can now
  define for the whole distributed system
• Its configurations
• We want it to be the state of all processes and
  the network
• Its initial configurations
• We want it to be all possible configurations
  where every local algorithm is in its start state
  and an empty network
• Its transitions, and
• We want each local algorithm state event (send,
  receive, internal) be a configuration transition
  in the distributed system

26
Configurations of the distributed system

• A distributed system running on processes p1, ,
  pn, where each pi is running a local algorithm
• ( Zpi, Ipi, Ipi,Spi,Rpi ), is modeled by an
  STS
• (C, ? , I)
• Where
• C (cp1, cp2, , cpn, M), where cpi?Zpi and M
  is a multiset of messages
• I.e., a configuration in the distributed system
  is a sequence of every processors current state,
  and a multiset of all the messages currently in
  transit

27
Initial configurations of the distributed system

• A distributed system running on processes p1, ,
  pn, where each pi is running a local algorithm
• ( Zpi, Ipi, Ipi,Spi,Rpi ), is modeled by an
  STS
• (C, ? , I)
• Where
• C (cp1, cp2, , cpn, M), where cpi?Zpi and M
  is a multiset of messages
• I (ip1, ip2, , ipn, M), where ipi?Ipi and M is
  empty ?
• I.e., the initial configuration of the
  distributed system can only be a configuration
  where every processor is in an initial state, and
  there are not messages in transit in the network

28
Transitions of the distributed system

• A distributed system running on processes p1, ,
  pn, where each pi is running a local algorithm
• ( Zpi, Ipi, Ipi,Spi,Rpi ), is modeled by an
  STS
• (C, ? , I)
• Where
• The transitions of the system are ? T1?T2 ?
  ?Tn
• Where for all 1?i?n, Ti is the set of all
  configuration transitions
• C1 ? C2 such that
• C1 (p1, , pi, , pn, M1)
• C2 (p1, , pj, , pn, M2)
• Where one the following is true
• (c, d)?Ipi such that M1M2 and pic and pjd
• (c, m, d)?Spi such that M2M1?m and pic and
  pjd
• (c, m, d)?Rpi such that M1M2?m and pic and
  pjd

29
Constraints

• We want to make sure that when the distributed
  system makes transitions, the following is
  preserved
• A process p can only receive a message after it
  has been sent to it by some other process
• A process p can only change from a state c to
  another state d if it is currently in state c
• Therefore, events can only happen if they are
  applicable

30
Applicable internal events

• Any internal event e(c,d)?Ipi is said to be
  applicable in an configuration C(cp1, , cpi, ,
  cpn, M) if cpic
• If event e is applied, we get e(C)(cp1, , d, ,
  cpn, M)
• Example, if there are 2 processors, p1, p2, and
• Set of states on p1 and p2 are Z1Z2s1, s2, s3
• Ip1 (s1,s3), (s3,s2), (s2, s3)
• Ip2 (s1,s2), (s2,s3), (s3, s1)
• The internal event (s2, s3)?Ip2 on p2 is
  applicable in all the following configurations
• ( s1, s2, M ), (s2, s2, M), and (s3, s2, M), for
  any multiset of messages M

31
Applicable send events

• Any send event e(c,m,d)?Spi is said to be
  applicable in an configuration C(cp1, , cpi, ,
  cpn, M) if cpic
• If event e is applied, we get e(C)(cp1, , d, ,
  cpn, M?m)
• Example, if there are 2 processors, p1, p2, and
• Set of states on p1 and p2 are Z1Z2s1, s2, s3
• Sp1 (s1, m, s3), (s3, n, s2), (s2, o, s3)
• Sp2 (s1, n, s2), (s2, m, s3), (s3, o, s1)
• The send event (s2, m, s3)?Sp2 on p2 is
  applicable in all the following configurations
• (s1, s2, M), (s2, s2, M), and (s3, s2, M), for
  any multiset of messages M

32
Applicable receive events

• Any receive event e(c,m,d)?Rpi is said to be
  applicable in an configuration C(cp1, , cpi, ,
  cpn, M) if cpic and m?M
• If event e is applied, we get e(C)(cp1, , d, ,
  cpn, M-m)
• Example, if there are 2 processors, p1, p2, and
• Set of states on p1 and p2 are Z1Z2s1, s2, s3
• Rp1 (s1, m, s3), (s3, n, s2), (s2, o, s3)
• Rp2 (s1, n, s2), (s2, m, s3), (s3, o, s1)
• The receive event (s2, m, s3)?Rp2 on p2 is
  applicable in all the following configurations
• (s1, s2, M), (s2, s2, M), and (s3, s2, M), for
  every multiset of messages M which contains the
  message m destined to p2

33
Synchronous message passing

• we now know how to model asynchronous message
  passing
• Synchronous message passing is similar, but
  easier. Read the book!

34
Fairness?

• Notice several events at different processes
  might be applicable in a given state
• Which one should be applied first?
• If e1, e2, e3 are applicable at a given time,
  should we assume they should be applied one after
  another e1, e2, e3?
• Maybe such an assumption would be too strong? We
  would like our model to not assume too much!
• We definitely do not want a have a stupid system
  where event e2 is always applicable, but never
  applied!
• In other words, we sometimes want to assume some
  fairness in the possible executions of our model

35
Weakly Fair and Strongly Fair executions

• An execution is weakly fair, if it is guaranteed
  that it can never happen that an event it
  applicable infinitely many times after each other
  (consecutively), without ever occurring in the
  execution
• An execution is strongly fair, if it is
  guaranteed that it can never happen that an event
  it applicable infinitely many times (not
  necessarily consecutively), without ever
  occurring in the execution

36
Proving correctness of an algorithm

• Given our formal model, we can now proof that an
  algorithm is correct
• Most interesting properties in distributed
  systems fall into one of these two categories
• Safety requirements
• Liveness properties

37
Safety and Liveness

• A safety requirement requires that some property
  holds in every execution in each reachable
  configuration
• A liveness requirement requires that some
  property will hold in every execution for some
  configuration which is reachable

38
Assertional Verification

• We will make an assertion of a property which we
  are interested to proof
• An assertion is a predicate on configurations of
  the distributed system
• I.e. a function which takes a configuration as
  input and returns either true or false
• P(?) is true or false

39
Safety Properties as assertions

• We proof a safety property by showing that its
  assertion is always true
• For an STS S(C,?, I) we write P?Q to denote
  that for each transition ??? in S we have that
• if P(?)true, then Q(?)true

40
An invariant property

• We say that a property P is invariant in S(C,?,
  I) if
• For all initial configurations ??I, P(?)true
• P?P for all configurations
• Theorem
• If P is invariant in S, then P is true in each
  configuration in every execution
• Proof
• For any execution in S, (?1, ?2, ?3), then we
  know by the definition of an execution that ?1 is
  an initial configuration, and by condition 1 of
  an invariant P(?1) is true
• By induction, condition 2 of an invariant gives
  that P(?k) is true for every k?2 ?

41
Liveness properties as Assertions

• Let term be an assertion which is only true if a
  configuration is terminal, and false otherwise
• To show a liveness property P, we want P to be
  true for some configuration in every execution
• We say that a system S terminates properly if
  term is true, then also P is true
• A partial order (W, lt) is well-founded if there
  is no infinite w1gtw2gtw3, where wi?W
• Example all positive natural numbers

42
Norm functions

• A norm function from the set of configurations C
  to a well-founded set W is a function f such that
  if ??? , for ?, ??C, then f(?)gtf(?) or P(?)true
• Example of a norm function for property P
• Configurations CX0, X1, X2, X3
• Configuration transitions ? X2?X1, X2?X0,
  X0?X3
• Norm function f X2?100, X1?50, X2?100, X0?1
• It is a norm function for this system because
• X2?X1, and f(X2)100 gt f(X1)50
• X2?X0, and f(X2)100 gt f(X0)1
• X0?X3, and P(X3)true

43
Proving liveness

• Theorem
• In a transition system S, which properly
  terminates for P, and a norm function f for P
  exists, then P is a liveness property which will
  be true for some configuration in every execution

44
Summary

• Distributed systems can be formally modeled by
  state transition systems STS
• Asynchronous message passing
• Synchronous message passing
• The model can be used to
• Prove impossibility results, i.e. statements
  about all possible algorithms in the system
• Prove that a certain property always holds
  (safety)
• Prove that a certain property will hold in every
  execution for some configurations

45
State transition system - example

• Example algorithm Using graphs
• X0
• while (Xlt3) do
• X X 1
• endwhile
• X1
• Formally
• States X0, X1, X2
• Possible transitions X0?X1, X1?X2, X2?X1
• Start states X0

X1
start
X0
X2
46
State transition system - formally

• A STS is formally described as the triple
• (C, ? , I)
• Such that
• C is a set of states
• ? is a subset of C ? C, describing the possible
  transitions (? ? C ? C)
• I is a subset of C describing the initial states
  (I ? C)
• Note that the system may have several transitions
  from one state E.g. ? X2?X1, X2?X0

47
Local Algorithms

• The local algorithm will be modeled as a STS, in
  which the following three events happen
• A processor changes state from one state to a
  another state (internal event)
• A processor changes state from one state to a
  another state, and sends a message to the network
  destined to another processor (send event)
• A processor receives a message destined to it and
  changes state from one state to another (receive
  event)

48
Model of the distributed system

• Based on the STS of a local algorithm, we can now
  define for the whole distributed system
• Its configurations
• We want it to be the state of all processes and
  the network
• Its initial configurations
• We want it to be all possible configurations
  where every local algorithm is in its start state
  and an empty network
• Its transitions, and
• We want each local algorithm state event (send,
  receive, internal) be a configuration transition
  in the distributed system

49
Execution Example 1/9

• Lets do a simple distributed application,
• where a client sends a ping, and receives
• a pong from a server
• This is repeated indefinitely

p1 client
sendltping, p2gt
sendltpong, p1gt
p2 server
50
Execution Example 2/9

• M pingp2, pongp1
• (Zp1, Ip1, Ip1,Sp1,Rp1)
• Zp1 cinit, csent
• Ip1 cinit
• Ip1 ?
• Sp1 (cinit, pingp2, csent)
• Rp1 (csent, pongp1, cinit)
• (Zp2, Ip2, Ip2,Sp2,Rp2)
• Zp2 sinit, srec
• Ip2 sinit
• Ip2 ?
• Sp2 (srec, pongp1, sinit)
• Rp2 (sinit, pingp2, srec)

p1 client
sendltping, p2gt
sendltpong, p1gt
p2 server
51
Execution Example 3/9
p2 (server) (Zp2, Ip2, Ip2,Sp2,Rp2) Zp2
sinit, srec Ip2 sinit Ip2 ? Sp2 (srec,
pongp1, sinit) Rp2 (sinit, pingp2, srec)
p1 (client) (Zp1, Ip1, Ip1,Sp1,Rp1) Zp1
cinit, csent Ip1 cinit Ip1 ? Sp1
(cinit, pingp2, csent) Rp1 (csent, pongp1,
cinit)

• C (cinit, sinit, ?), (cinit, srec, ?), (csent,
  sinit, ?), (csent, srec, ?),
• (cinit, sinit, pingp2), (cinit, srec,
  pingp2), (csent, sinit, pingp2), (csent,
  srec, pingp2),
• (cinit, sinit, pongp1), (cinit, srec,
  pongp1), (csent, sinit, pongp1), (csent,
  srec, pongp1)
• ...
• I (cinit, sinit, ?)
• ? (cinit, sinit, ?) ? (csent, sinit, pingp2),
  
• (csent, sinit, pingp2) ? (csent, srec, ?),
• (csent, srec, ?) ? (csent, sinit, pongp1),
• (csent, sinit, pongp1) ? (cinit, sinit, ?)

52
Execution Example 4/9
p2 (server) (Zp2, Ip2, Ip2,Sp2,Rp2) Zp2
sinit, srec Ip2 sinit Ip2 ? Sp2 (srec,
pongp1, sinit) Rp2 (sinit, pingp2, srec)
p1 (client) (Zp1, Ip1, Ip1,Sp1,Rp1) Zp1
cinit, csent Ip1 cinit Ip1 ? Sp1
(cinit, pingp2, csent) Rp1 (csent, pongp1,
cinit)

• I (cinit, sinit, ?)
• ?(cinit, sinit, ?) ? (csent, sinit, pingp2),
• (csent, sinit, pingp2) ? (csent, srec, ?),
• (csent, srec, ?) ? (csent, sinit, pongp1),
• (csent, sinit, pongp1) ? (cinit, sinit, ?)
• E( (cinit, sinit, ?) , (csent, sinit, pingp2),
• (csent, srec, ?), (csent, sinit, pongp1),
• (cinit, sinit, ?), (csent, sinit, pingp2), )

p1 state cinit
p2 state sinit
53
Execution Example 5/9
p2 (server) (Zp2, Ip2, Ip2,Sp2,Rp2) Zp2
sinit, srec Ip2 sinit Ip2 ? Sp2 (srec,
pongp1, sinit) Rp2 (sinit, pingp2, srec)
p1 (client) (Zp1, Ip1, Ip1,Sp1,Rp1) Zp1
cinit, csent Ip1 cinit Ip1 ? Sp1
(cinit, pingp2, csent) Rp1 (csent, pongp1,
cinit)

• I (cinit, sinit, ?)
• ?(cinit, sinit, ?) ? (csent, sinit, pingp2),
• (csent, sinit, pingp2) ? (csent, srec, ?),
• (csent, srec, ?) ? (csent, sinit, pongp1),
• (csent, sinit, pongp1) ? (cinit, sinit, ?)
• E( (cinit, sinit, ?) , (csent, sinit, pingp2),
• (csent, srec, ?), (csent, sinit, pongp1),
• (cinit, sinit, ?), (csent, sinit, pingp2), )

p1 state cinit
p2 state sinit
54
Execution Example 6/9
p2 (server) (Zp2, Ip2, Ip2,Sp2,Rp2) Zp2
sinit, srec Ip2 sinit Ip2 ? Sp2 (srec,
pongp1, sinit) Rp2 (sinit, pingp2, srec)
p1 (client) (Zp1, Ip1, Ip1,Sp1,Rp1) Zp1
cinit, csent Ip1 cinit Ip1 ? Sp1
(cinit, pingp2, csent) Rp1 (csent, pongp1,
cinit)

• I (cinit, sinit, ?)
• ?(cinit, sinit, ?) ? (csent, sinit, pingp2),
• (csent, sinit, pingp2) ? (csent, srec, ?),
• (csent, srec, ?) ? (csent, sinit, pongp1),
• (csent, sinit, pongp1) ? (cinit, sinit, ?)
• E( (cinit, sinit, ?) , (csent, sinit, pingp2),
• (csent, srec, ?), (csent, sinit, pongp1),
• (cinit, sinit, ?), (csent, sinit, pingp2), )

p1 state csent
sendltping, p2gt
p2 state sinit
55
Execution Example 7/9
p2 (server) (Zp2, Ip2, Ip2,Sp2,Rp2) Zp2
sinit, srec Ip2 sinit Ip2 ? Sp2 (srec,
pongp1, sinit) Rp2 (sinit, pingp2, srec)
p1 (client) (Zp1, Ip1, Ip1,Sp1,Rp1) Zp1
cinit, csent Ip1 cinit Ip1 ? Sp1
(cinit, pingp2, csent) Rp1 (csent, pongp1,
cinit)

• I (cinit, sinit, ?)
• ?(cinit, sinit, ?) ? (csent, sinit, pingp2),
• (csent, sinit, pingp2) ? (csent, srec, ?),
• (csent, srec, ?) ? (csent, sinit, pongp1),
• (csent, sinit, pongp1) ? (cinit, sinit, ?)
• E( (cinit, sinit, ?) , (csent, sinit, pingp2),
• (csent, srec, ?), (csent, sinit, pongp1),
• (cinit, sinit, ?), (csent, sinit, pingp2), )

p1 state csent
recltping, p2gt
p2 state srec
56
Execution Example 8/9
p2 (server) (Zp2, Ip2, Ip2,Sp2,Rp2) Zp2
sinit, srec Ip2 sinit Ip2 ? Sp2 (srec,
pongp1, sinit) Rp2 (sinit, pingp2, srec)
p1 (client) (Zp1, Ip1, Ip1,Sp1,Rp1) Zp1
cinit, csent Ip1 cinit Ip1 ? Sp1
(cinit, pingp2, csent) Rp1 (csent, pongp1,
cinit)

• I (cinit, sinit, ?)
• ?(cinit, sinit, ?) ? (csent, sinit, pingp2),
• (csent, sinit, pingp2) ? (csent, srec, ?),
• (csent, srec, ?) ? (csent, sinit, pongp1),
• (csent, sinit, pongp1) ? (cinit, sinit, ?)
• E( (cinit, sinit, ?) , (csent, sinit, pingp2),
• (csent, srec, ?), (csent, sinit, pongp1),
• (cinit, sinit, ?), (csent, sinit, pingp2), )

p1 state csent
sendltpong, p1gt
p2 state sinit
57
Execution Example 9/9
p2 (server) (Zp2, Ip2, Ip2,Sp2,Rp2) Zp2
sinit, srec Ip2 sinit Ip2 ? Sp2 (srec,
pongp1, sinit) Rp2 (sinit, pingp2, srec)
p1 (client) (Zp1, Ip1, Ip1,Sp1,Rp1) Zp1
cinit, csent Ip1 cinit Ip1 ? Sp1
(cinit, pingp2, csent) Rp1 (csent, pongp1,
cinit)

• I (cinit, sinit, ?)
• ?(cinit, sinit, ?) ? (csent, sinit, pingp2),
• (csent, sinit, pingp2) ? (csent, srec, ?),
• (csent, srec, ?) ? (csent, sinit, pongp1),
• (csent, sinit, pongp1) ? (cinit, sinit, ?)
• E( (cinit, sinit, ?) , (csent, sinit, pingp2),
• (csent, srec, ?), (csent, sinit, pongp1),
• (cinit, sinit, ?), (csent, sinit, pingp2), )

p1 state cinit
recltpong, p1gt
p2 state sinit
58
Applicable events

• Any internal event e(c,d)?Ipi is said to be
  applicable in an configuration C(cp1, , cpi, ,
  cpn, M) if cpic
• If event e is applied, we get e(C)(cp1, , d, ,
  cpn, M)
• Any send event e(c,m,d)?Spi is said to be
  applicable in an configuration C(cp1, , cpi, ,
  cpn, M) if cpic
• If event e is applied, we get e(C)(cp1, , d, ,
  cpn, M?m)
• Any receive event e(c,m,d)?Rpi is said to be
  applicable in an configuration C(cp1, , cpi, ,
  cpn, M) if cpic and m?M
• If event e is applied, we get e(C)(cp1, , d, ,
  cpn, M-m)

59
Order of events

• The following theorem shows an important result
• The order in which two applicable events are
  executed is not important!
• Theorem
• Let ep and eq be two events on two different
  processors p and q which are both applicable in
  configuration ?. Then ep can be applied to eq(?),
  and eq can be applied to ep(?).
• Moreover, ep(eq(?)) eq(ep(?) ).

60
Order of events

• To avoid a proof by cases (339 cases) we
  represent all three event types in one
  abstraction
• We let the quadtuple (c, X, Y, d) represent any
  event
• c is the initial state of the processor
• X is a set of messages that will be received by
  the event
• Y is a set of message that will be sent by the
  event
• d is the state of the processor after the event
• Examples
• (cinit, ?, ping, csent) represents a send event
• (sinit, pong, ?, srec) represents a receive
  event
• (c1r, ?, ?, c2) represents an internal event
• Any such event ep(c, X, Y, d), at p, is
  applicable in a state ? , cp,, M if and only
  if cpc and X?M.

61
Order of events

• Proof
• Let epc, X, Y, d and eqe, Z, W, f and ?
  , cp, , cq,, M
• As both ep and ep are applicable in ? we know
  that cpc, cqe, X?M, and Z?M.
• eq(?), cp, , f,, (M-Z)?W,
• cp is untouched and cpc, and X? (M-Z)?W as
  X?Z?, hence ep is applicable in eq(?)
• Similar argument to show that eq is applicable in
  ep(?)

62
Order of events

• Proof
• Lets proof ep(eq(?)) eq(ep(?) )
• eq(?), cp, , f,, (M-Z)?W
• ep(eq(?)), d, , f,, (((M-Z)?W)-X)?Y
• ep(?), d, , cq,, (M-X)?Y
• eq(ep(?)), d, , f,, (((M-X)?Y)-Z)?W
• (((M-Z)?W)-X)?Y (((M-X)?Y)-Z)?W
• Because X?Z?, W?X?, Y?Z?
• Both LHS and RHS can be transformed to ((M ?W
  ?Y)-Z)-X
• ?

63
Exact order does not always matter

• In two cases the theorem does not apply
• If pq, i.e. when the events occur on different
  processes
• They would not both be applicable if they are
  executed out of order
• If one is a sent event, and the other is the
  corresponding receive event
• They cannot be both applicable
• In such cases, we say that the two events are
  causally related!

64
Causally Order

• The relation H on the events of an execution,
  called causal order, is defined as the smallest
  relation such that
• If e occurs before f on the same process, then e
  H f
• If s is a send event and r its corresponding
  receive event, then s H r
• H is transitive.
• I.e. If a H b and b H c then a H c
• H is reflexive.
• I.e. If a H a for any event a in the execution
• Two events, a and b, are concurrent iff a H b
  and b H a holds

65
Example of Causally Related events
Concurrent Events
Causally Related Events

• Time-space diagram

p1
p2
p3
time
Causally Related Events
66
Equivalence of Executions Computations

• Computation Theorem
• Let E be an execution E(?1, ?2, ?3), and V be
  the sequence of events V(e1, e2, e3) associated
  with it
• I.e. applying ek(?k )?k1 for all k1
• A permutation P of V that preserves causal order,
  and starts in ?1, defines a unique execution with
  the same number of events, and if finite, P and
  Vs final configurations are the same
• P(f1, f2, f3) preserves the causal order of V
  when for every pair of events fi H fj implies iltj

67
Equivalence of executions

• If two executions F and E have the same
  collection of events, and their causal order is
  preserved, F and E are said to be equivalent
  executions, written FE
• F and E could have different permutation of
  events as long as causality is preserved!

68
Computations

• Equivalent executions form equivalence classes
  where every execution in class is equivalent to
  the other executions in the class.
• I.e. the following always holds for executions
• is reflexive
• I.e. If a a for any execution
• is symmetric
• I.e. If ab then ba for any executions a and b
• is transitive
• If ab and bc, then ac, for any executions a,
  b, c
• Equivalence classes are called computations of
  executions

69
Example of equivalent executions
Same color Causally related
p1

• All three executions are part of the same
  computation, as causality is preserved

p2
p3
time
p1
p2
p3
time
70
Two important results (1)

• The computation theorem gives two important
  results
• Result 1
• There is no distributed algorithm which can
  observe the order of the sequence of events (that
  can see the time-space diagram)
• Proof
• Assume such an algorithm exists. Assume process p
  knows the order in the final configuration
• Run two different executions of the algorithm
  that preserve the causality.
• According to the computation theorem their final
  configurations should be the same, but in that
  case, the algorithm cannot have observed the
  actual order of events as they differ ?

71
Two important results (2)

• Result 2
• The computation theorem does not hold if the
  model is extended such that each process can read
  a local hardware clock
• Proof
• Similarly, assume a distributed algorithm in
  which each process reads the local clock each
  time a local event occurs
• The final configuration of different causality
  preserving executions will have different clock
  values, which contradicts the computation theorem
• ?

72
Lamport Logical Clock (informal)

• Each process has a local logical clock, which is
  kept in a local variable ?p for process p, which
  is initially 0
• The logical clock is updated for each local event
  on process p such that
• If an internal or send event occurs
• ?p ?p1
• If a receive event happens on a message from
  process q
• ?p max(?p, ?q)1
• Lamport logical clocks guarantee that
• If aHb, then ?p?q,
• where a and be happen on p and q

73
Example of equivalent executions
p1
0
1
3
4
2
p2
0
4
5
p3
0
1
6
time
74
Vector Timestamps useful implication

• Each process p keeps a vector vpn of n
  positions for system with n processes, initially
  vpp1 and vpi0 for all other i
• The logical clock is updated for each local event
  on process p such that
• If any event
• vp pvp p1
• If a receive event happens on a message from
  process q
• vp x max(vpx, vqx), for 1xn
• We say vpvq y iff vpxvqx for 1xn
• If not vpvq and not vqvp then vp is concurrent
  with vq
• Lamport logical clocks guarantee that
• If vpvq then aHb, and aHb, then vpvq,
• where a and be happen on p and q

75
Example of Vector Timestamps
p1
1,0,0
2,0,0
4,0,0
5,0,0
3,0,0
p2
0,1,0
4,2,0
4,3,0
p3
0,0,1
0,0,2
4,3,3
time
This is great! But cannot be done with smaller
vectors than size n, for n processes
76
Useful Scenario what is most recent?
p1
1,0,0
2,0,0
4,0,0
3,0,0
5,0,0
p2
5,2,0
5,3,0
0,1,0
p3
0,0,1
5,0,2
5,0,3
time
p2 examines the two messages it receives, one
from p1 4,0,0 and one from p2 5,0,3 and
deduces that the information from p1 is the
oldest (4,0,05,0,3).
77
Summary

• The total order of executions of events is not
  always important
• Two different executions could yield the same
  result
• Causal order matters
• Order of two events on the same process
• Order of two events, where one is a send and the
  other one a corresponding receive
• Order of two events, that are transitively
  related according to above
• Executions which contain permutations of each
  others event such that causality is preserved are
  called equivalent executions
• Equivalent executions form equivalence classes
  called computations.
• Every execution in a computation is equivalent to
  every other execution in its computation
• Vector timestamps can be used to determine
  causality
• Cannot be done with smaller vectors than size n,
  for n processes