ZYZZYVA: SPECULATIVE BYZANTINE FAULT TOLERANCE

1
ZYZZYVASPECULATIVE BYZANTINEFAULT TOLERANCE

• R.Kotla, L. Alvisi, M. Dahlin, A. Clement and
  E. WongU. T. Austin

Best Paper Award at SOSP 2007
2
Motivation

• Why implement Byzantine Fault-Tolerant
  replication?
• Increasing value of data and decreasing cost of
  hardware
• More non-stop-fail behaviors than believed
• BFT is becoming cheaper
• Cost of 3-way non-BFT replication close to cost
  of BFT replication

3
Zyzzyva (I)

• Uses speculation to reduce the cost of BFT
  replication
• Primary replica proposes order of client requests
  to all secondary replicas (standard)
• Secondary replicas speculatively execute the
  request without going through an agreement
  protocol to validate that order (new idea)

4
Zyzzyva (II)

• As a result
• States of correct replicas may diverge
• Replicas may send diverging replies to client
• Zyzzyvas solution
• Clients detect inconsistencies
• Help convergence of correct replicas to a single
  total ordering of requests
• Reject inconsistent replies

5
How?

• Clients observe a replicated state machine
• Replies contain enough information to let clients
  ascertain if the replies and the history are
  stable and guaranteed to be eventually committed
• Replicas have checkpoints

6
Byzantine agreement (I)

• No solution for less than four entities

7
Byzantine agreement (II)

• To achieve agreement in the presence of f failed
  nodes (traitors) we need
• 3f 1 entities

8
Practical BFT (I)

• Practical Byzantine Fault-Tolerant protocol
  (PBFT) Castro and Liskov 1999

9
Practical BFT (II)
Replicas decide on correct ordering
10
Practical BFT (III)

1. Client sends signed request to primary replica
2. Primary assigns a sequence number to the request
   and sends to all other replicas aPRE-PREPARE
   message
3. Secondary replicas validate the message and send
   a PREPARE message to all replicas
4. Replicas that can collect 2f PREPARE messages
   send a COMMIT message to all replicas
5. Replicas that can collect 2f 1 COMMIT message
   send a REPLY to the client

11
A shortened version
Faster agreement is achieved thanks toa more
complex view change protocol
12
The explanation (I)

• "No replicated service that uses the traditional
  view change protocol can be live without an
  agreement protocol that includes both the prepare
  and commit full exchanges"
• "The traditional view change protocol lets
  correct replicas commit to a view change and
  become silent in a view without any guarantee
  that their action will lead to the view change."

13
The explanation (II)

• Zyzzyva
• Adds an extra phase to its view change protocol
• Guarantees that a correct replica will not
  abandon a view unless every other correct
  replica does it

14
Zyzzyva Agreement (I)

• Common case no faulty replicas

15
Explanations

• Secondary replicas assume that
• Primary replica gave the right ordering
• All secondary replicas will participate in
  transaction
• Initiate speculative execution
• Client receives 3f 1 mutually consistent
  responses

16
Zyzzyva Agreement (II)

• With a faulty replica

17
Explanations (I)

• Client receives 3f mutually consistent responses
• Gathers at least 2f 1 mutually consistent
  responses
• Distributes a commit certificate to the replicas
• Once at least 2f 1 replicas acknowledge
  receiving a commit certificate, the client
  considers the request completed

18
Explanations (II)

• If enough secondary replicas suspect that the
  primary replica is faulty, a view change is
  initiated and a new primary elected

19
Comparison with traditional solutions
20
State maintained at each replica
21
Explanations (I)

• Each replica maintains
• A history of the requests it has executed
• A copy of the max commit certificate it has
  received
• Let it distinguish between committed history and
  speculative history

22
Explanations (II)

• Each replica constructs a checkpoint every
  CP_INTERVAL requests
• It maintains one stable checkpoint with a
  corresponding stable application state snapshot
• It might also have up to one speculative
  checkpoint with its corresponding speculative
  application state snapshot

23
Explanations (III)

• Checkpoints and application state become
  committed through a process similar to that of
  earlier BFT agreement protocols
• Replicas send signed checkpoint messages to all
  replicas when they generate a tentative
  checkpoint
• Commit checkpoint after they collect f 1
  signed matching checkpoint messages

24
View change sub-protocol (I)
25
Explanations

• Two-phase protocol
• Elects a new primary
• Guarantees that it will not introduce any changes
  in a history that has already completed at a
  correct client

26
Performance throughput
27
Comments

• Zyzzyva-5 is a special version of Zyzziva
  requiring more replicas but having a lower
  overhead

28
Performance latency
29
Scalability peak throughputs
30
CONCLUSIONS

• Systematically exploiting speculative execution
  results in a protocol much faster than
  conventional BFT agreement protocols.

• Observe that Zyzzyva is optimized for the most
  frequent case but provides the correct result in
  all cases
• A good rule to follow