Verification Codes

1
Verification Codes

• Michael Luby, Digital Fountain, Inc.
• Michael Mitzenmacher
• Harvard University and Digital Fountain, Inc.

2
Motivation

• LDPC (Low-Density Parity-Check) codes
• Linear time and near optimal performance.
• For erasures Packet-level code, where basic op
  is XORing packets. Very fast.
• For errors Bit-level code, where basic op is
  probabilistic calculation or XORing bits. Slow.
• Bit level unsuitable for some applications.
• Packet-level codes useful as outer codes.
• Can we design packet-level LDPC codes for
  channels with errors?

3
Design Steps

• LDPC codes for symmetric q-ary channels, large q
• Use that errors are random in a large space.
• Tree analysis.
• Adversarial errors to random errors.
• Code scrambling, independently discovered by
  L94,GLD01.
• Implementation details.
• Packet numbers.

4
Setup

• View code as one layer bipartite graph.
• Variable nodes on left.
• Check nodes on right.
• Codewords satisfy sum of all neighbors of a
  check node is 0.
• Note sum allows general q. For bits/ packets,
  sum is XOR.

A
B
H
C
I
D
J
E
K
F
G
5
Verification

• A message node is verified if it is believed to
  have the right value, with high probability.
• For symmetric q-ary channel, a check node
  verifies its neighbors if they sum to 0.

A
B
H
C
If A B C 0, then H verifies A, B, C.
6
Verification Argument

• Claim If errors are random, probability of a
  false verification at each step is at most
  1/(q-1).
• Proof Consider the last erroneous message node.
  It must take on the precise value to cause a
  false verification.
• For large q, verifications are correct with high
  probability.

A
B
H
C
7
Correction Mechanism

• If B and C are verified, H can correct and verify
  value for A.
• Corrections possible when all but one neighbor of
  a check node has been verified.
• If verifications correct with high probability,
  so are corrections.

A
B
H
C
8
Message Passing Algorithm 1

• Message nodes may be verified or unverified.
• Initially all unverified.
• Message nodes have a value.
• Initially received value.
• In parallel message nodes send value, state.
• If all neighbors of a check node sum to 0, check
  node verifies all neighbors.
• If all but one neighbor of a check node is
  verifed, the check node can correct and verify
  the value of the remaining node.
• In parallel check nodes send appropriate
  messages.
• Repeat.

9
Analysis

• Error probability from incorrect verification
  bounded by (verification steps)/(q-1). Small
  for large q.
• Ignored hereon in the analysis.
• End goal all message nodes are verified.
• LDPC tree analysis yields equations.

10
Degree Sequence Functions

• Left Side
• fraction of edges of degree i on the left.
• Right Side
• fraction of edges of degree i on the right.

11
Tree Based Analysis
After j rounds
Left
Prnot verified, correct
Prnot verified, incorrect
Prverified
Right
Assume verified correct.
Left
12
Tree Based Analysis

• Defining equations.
• English to be correct and unverified after round
  j1, must be
• Correct initially.
• Each check node under you must have one incorrect
  node under it after round j.
• Probability , z Pra check node
  has no incorrect neighbor after round j
• .

13
Tree Based Analysis

• Defining equations.
• To be incorrect after round j1, must be
• Incorrect initially.
• Each check node under you must have one
  unverified node under it after round j.
• Equivalently
• Want

14
Bounds

• Finding good polynomials l and r is difficult
  non-linear optimization problem.
• From previous work on LDPCs, for code rate R have
  l and r such that
• Plugging in this l and r yield codes that will
  handle fraction of errors up to
• Good for low rate codes.

15
Proof
Let x bj. Then for
Now use
Suffices that
16
Improved algorithm
A
B
H
Incorrect
C
I
Verified
D
Verified
E
F
17
Improved Algorithm

• Each check node also sends a proposed value to
  each message node.
• Equal to 0 - sum of other neighbors.
• If a message node receives two of the same
  proposed values, assumes it is the correct value
  and sets itself to verified.
• Important no cycles of length 4.
• Again, small probability of incorrect
  verification.
• Similar limiting equations derivable.

18
Reed-Solomon Verification
A
B
RS1
RS2
C
D
Tradeoff RS check nodes yield more
redundancy, for more powerful check nodes.
19
Reed-Solomon Verification

• Check nodes consist of a Reed-Solomon code over
  the message nodes.
• Example check nodes consists of two points.
• So check node of degree d defines a degree d - 1
  polynomial.
• Now check nodes can
• Correct any single error among neighbors.
• Correct two errors if all other neighbors are
  verified.

20
Bounds

• Equations in terms of l and r derivable.
• Finding good polynomials l and r is a difficult
  non-linear optimization problem.
• Consider case where check nodes can correct if at
  most 1 neighbor in error.
• LDPC erasure codes correct if 1 at most 1
  neighbor erased.
• Equivalency implies can correct
  fraction of errors.
• Better using additional power of check nodes.

21
Comparison Point

• Standard Reed-Solomon codes can handle error rate
  of up to (1-R)/2.
• Different model worst case errors, not random.
• Also additional techniques List Decoding.
• Verification codes can provably handle higher
  error rates, with high probability.
• Low rates using simple verification.
• High rates using verification 2 RS nodes.

22
Code Scrambling

• Using shared (pseudo)-randomness, errors can
  appear random instead of worst case.
• Assume a suitably oblivious adversary.
• Permute which data in which message packets, so
  adversary does not know where errors are
  introduced.
• Replace jth packet data x with cjx dj for
  random values cj and dj inversion causes
  adversarial error to look random.
• Symmetric q-ary code sufficient against a
  suitably oblivious channel.

23
Code Scrambling Example

• Consider modulo 61.
• If adversary does not know c, d the inverted
  values looks random.

Data c d Sent Error
Inverted
38
39
18
23
7
11
13
0
8
5
11
55
24
Implementation Details

• Packets must have ID bits.
• If data can be corrupted so might ID bits.
• These errors can be absorbed in data errors.
• Bad ID will either be an erased packet
  (equivalent to an error) or will cause another
  data packet to be in error.

25
Additional Work

• Verification codes based on LT (Luby Transform)
  codes.
• Extended verification codes unverified nodes
  send speculative list of values.
• To appear in the full paper.

26
Related Work

• Gallager mentions LDPC codes for symmetric q-ary
  channels back in 1960s.
• But did not see the possibilities of verification
  for large q.
• LDPC codes over GF(q), Davey and MacKay.
• Meant for small q.
• Code scrambling.
• Applied primarily to Reed-Solomon codes.

27
Conclusions

• Verification paradigm leads to simple codes for
  random errors over large alphabets.
• Potentially useful as an outer code.
• Inner code say an LDPC code on a packet of bits.
• Faulty decoding leads to arbitrary packet errors.
• Useful when avoiding bit-level calculations.
• Codes in software vs. hardware.