Principles of Reliable Distributed Systems Recitation 2: Broadcast Services

1
Principles of Reliable Distributed Systems
Recitation 2 Broadcast Services

• Spring 2009
• Alex Shraer

2
Broadcast Service for Replication

• Primitives broadcast(m), deliver(m).
• For simplicity, assume m is unique.

Application
Application
broadcast
broadcast
deliver
deliver
Broadcast Algorithm
Broadcast Algorithm
receive
send
receive
send
Network
3
Reliable Broadcast Specifications

• Validity if a correct process broadcasts m then
  all correct processes eventually deliver m
• Agreement if a correct process delivers m then
  all correct processes eventually deliver m
• Uniform Agreement if any process delivers m then
  all correct processes eventually deliver m
• Integrity m is delivered by a correct process at
  most once, and only if it was previously broadcast

4
Reliable Broadcast - Quiz

• What happens if a process fails during the
  broadcast of a message?
• Does a message delivery by a faulty process
  require the delivery of this message by correct
  processes?

5
FIFO Broadcast

• Why is FIFO important?
• FIFO Order If a process broadcasts a message m
  before it broadcasts a message m, then no
  correct process delivers m unless it has
  previously delivered m.
• FIFO Broadcast Reliable broadcast FIFO Order
• Alternative definition of FIFO Order?
• all messages broadcast by the same process are
  delivered to all processes in the order they are
  sent
• Quiz Are these definitions equivalent?

6
Example

• Also, this alternative definition forces faulty
  processes to deliver messages. (impossible)

7
Causal Broadcast

• Why is causality important?
• Event e causally precedes event f (e?f) iff
• a process executes both e and f, in that order,
  or
• e is the broadcast of some message m and f is the
  delivery of m, or
• There is an event h, such that e?h and h?f.
• Causal Order If the broadcast of a message m
  causally precedes the broadcast of a message m,
  then no correct process delivers m unless it has
  previously delivered m.
• Causal Broadcast Reliable broadcast Causal
  order

8
Atomic Broadcast and Uniformity

• Why would we want more than Causal Broadcast?
• Atomic Broadcast Reliable Broadcast Total
  Order
• Total Order if correct processes p and q both
  deliver messages m and m, then p delivers m
  before m if and only if q delivers m before m.

9
Broadcast Primitives
10
Uniformity

• Agreement, Integrity and Order place no
  restrictions on the behavior of faulty processes.
• Uniform limit the behavior of faulty processes
• Example 1 Agreement allows a faulty process to
  deliver a message that is never delivered by
  correct processes
• Uniform Agreement If a process (whether correct
  or faulty) delivers a message m, then all correct
  processes eventually deliver m.
• Example 2 Integrity allows a faulty process to
  deliver a message more than once, and to deliver
  messages out of thin air
• Uniform Integrity For any message m, every
  process (whether correct or faulty) delivers m at
  most once, and only if some process broadcast m.
• Likewise, we can strengthen the Order properties
• Uniform FIFO Order If a process broadcasts a
  message m before it broadcasts a message m, then
  no process (whether correct of faulty) delivers
  m unless it has previously delivered m.
• Uniform Causal Order If the broadcast of a
  message m causally precedes the broadcast of a
  message m, then no process (whether correct or
  faulty) delivers m unless it has previously
  delivered m.
• Uniform Total Order if any processes p and q
  (whether correct or faulty) both deliver messages
  m and m, then p delivers m before m iff q
  delivers m before m.

11
Crash Failures

• Suppose processes are only subject to crash
  failures
• They operate correctly up to the time they crash
  (by definition).
• Can we assume that the message deliveries that a
  process makes before crashing are always
  correct (consistent with those of correct
  processes)?
• No

12
Crash Failures (contd)

• Coordinator-based algorithm
• When a process intends to broadcast a message m,
  it first sends m to a coordinator.
• The coordinator delivers messages in the order in
  which it receives them, and periodically informs
  the other processes of this message delivery
  order.
• Other processes deliver messages according to
  this order.
• If the coordinator crashes, another process takes
  over as coordinator.

13
Crash Failures (contd)

• The algorithm satisfies the specification Atomic
  Broadcast
• Suppose a coordinator delivers m before m and
  crashes.
• A new coordinator could think m is before m.
• All correct processes follow the new coordinator
• Thus, the old coordinator delivered messages out
  of order before it crashed.
• Inconsistency can occur even when there are only
  crash failures.
• Protocols should explicitly prevent inconsistency
  even when there are only crash failures

14
LTS Broadcast Algorithm - code for process pi

• Logical Clock Assignment
• TSj ? 0, ?j0,,n
• pending ? empty
• broadcast (m)
• TSi ? TSi 1
• send (m, ?TSi, i?) to all
• upon receive (m, ?t, j?)
• TSj ? t
• add (m, ?t, j?) to pending
• TSi ? max (TSi, t) 1
• Delivery Rule
• let (m, ?t, j?) be the entry in pending
  with the smallest ?t, j?
• if ?t, j? ? ?TSk,k? ?k0,n then
• deliver (m)
• remove (m, ?t, j?) from pending

15
Example Exam Question
2
7
3
8
0
?6,2?
?1,2?
?3,1?
12
12
4
5
0
1
7
7
6
?3,3?
7
10
0
Delivery according to LTS
2
7
3
4

• ??? ???? ?????? ?? ?? ??????? ???? ????? ?? ???
  ?-causality (????? happens-before)?
• ???? ?? ??? logical clock ??? ??? ???? ?????.
  ???? ?? ??? ?-LTS ?????? ??? ?????.
• ????? ??? (????? t) ?? ????? ???? delivery
  ??????? m1 ? m3. ?? ????? ??????? ?? ?????
  delivery ????? ??? ?????.

16
Vector Clocks

• At process pi, on broadcast(m)
• VCi VCi1
• use reliable broadcast to send m with VC to all
• deliver m locally
• Upon receive m
• place in message buffer
• Deliver m from pj from buffer if
• VCj m.VCj 1
• forall k?j VCk m.VCk
• Upon deliver
• VCj VCj 1

FIFO
VCj is the number of messages of pj that
causally precede pis subsequent messages
17
Example Exam Question Cont.
0,0,0
0,1,0
1,1,0
1,2,1
1,1,0
1,1,0
1,2,1
0,1,0
3
4
0,0,0
1,1,1
0,1,1
1,2,1
1
7
0,1,0
2
7
0,1,1
0,0,0
Delivery according to VC
1,2,1
0,1,0
1,1,1
0,1,1

• ???? ?? ?-Vector Clock ???? ????? ??? ????? ??
  ????? ??????? ???.
• ????? ??? (????? t) ?? ????? ???? delivery
  ??????? m1 ? m3. ?? ????? ??????? ?? ?????
  delivery ????? ??? ?????.