FEC / Erasure Codes techniques and applications

1
FEC / Erasure Codes techniques and applications

• Noam Singer

2
Index
BEC
Random XOR
Parallel Download
Tornado Codes
LT Codes
Erasure
Bulk Data
RT Oblivious
Matrix
RMDP
Polynomial
Randomized
Digital Fountain
3
BEC Binary Erasure Channel

• Message is packetized.
• Packets may be corrupted or lost during
  transmissions.
• Detection using CRC with high probability.
• Using cryptographic authentication may prevent
  packet modification.

4
Handling corruptions

• Ignore lost packets and errors.
• Use ARQ techniques
• Possible only if there exist a feed-back channel.
• May result with long round-trip delays.
• For multicast it is un-scalable.
• Use erasure correcting codes to restore lost
  packets.

S
R
5
Erasure codes
6
Erasure codes

• Handle packet losses.
• Any errored packet is considered lost.
• Encode and send more redundant packets with which
  the decoder may restore some lost packets.
• The lost packets IDs must be known to the decoder.

0
1
2
3
4
5
6
7
8
9
10
7
Definitions

• Data original k packets which we want to
  transmit.
• Code Encoding of the data to nk packets which
  are transmitted.
• Optimal encoding (n,k) An encoding of k data
  packets to n code packets, to which from every
  subset of k packets we can reconstruct the
  original data.
• Redundency rn-k
• Encoding rate Rk/n
• Stretch factor cn/k

n
k
k
r
8
Optimal encoding (n,k)
k message
9
Trivial Optimal Codes

• Copying 1 packet to n packets is an (n,1)
  encoding.
• Calculating 1 redundant packet as the XOR of all
  k packets, a (k1,k) encoding.

P
C
C
C
C
C
C
C
P1
P2
P3
P4
P5
P6
P7
XOR
10
Non-optimal codes

• Some codes require slightly more than k packet to
  restore the original data.
• e is called Encoding Inefficiency if k(1e)
  packets are required for decoding on the average.
• Decoding efficiency is 1/(1e)
• Optimal encoding has decoding efficiency 1

k
k
r
k
11
Decoding limitations

• When receiving less than k packets, no decoder
  exists, which can reconstruct the entire original
  message.This is true because there can be no
  compression in our model.
• ?Any (n,k) optimal encoding, cant correct more
  than n-k losses.

???
k
r
k
12
Matrix erasure code

• Well place packets in an mxm matrix.
• Well add redundant XOR packets of the data in
  each row and column.
• Well also protect the XOR row and column with
  another XOR packet.
• The data size is km2
• The code size is

0 0 1 1 0
1 0 0 1 0
1 1 1 0 1
1 0 0 1 0
1 1 0 1 1
13
Matrix correction

• All packets may be lost, even the new redundant
  packets.
• Algorithm
• Find a line or a column with only one error, and
  fix lost packets to the XOR of all correct
  packets.
• We continue until we dont find any.
• Decoding complexity O(e)

0 0 1 1 0
1 0 0 1 0
1 1 1 0 1
1 0 0 1 0
1 1 0 1 1
14
Correction sample
? 0 1 1 ?
1 ? ? 1 0
1 ? ? 0 ?
1 0 0 1 0
1 1 0 1 1
0
0
1
15
Unresolved situations

• A knot is a subset of the errors, in which every
  row/column has at least two errors.
• Knots can not be resolved.
• Such a knots will always exist for egt2m1
• Decoding limitations
• Less than 4 errors can always be fixed.
• Up to 2m1 errors might be fixed
• More than 2m1 errors can never be fixed.

16
d-dimensional matrix

• We can expand the matrix model to a
  d-dimensional matrix, in which for each
  d-dimensional column we calculate a XOR packet.

17
d-dimensional limitations

• Code size (m1)d
• Stretch factor (11/m)d
• Decoding complexity O(ed)
• Decoding limitations
• Less than 2d errors can always be fixed.
• Up to (m1)d-md errors might be fixed
• More than (m1)d-md errors can never be fixed.

18
Polynomial codes

• We use a polynomial P(x) of degree k-1 over a
  finite field, which is large enough.
• The data Y0Yk-1 is the value of P(x) for x0k-1
• The Redundant information YkYn-1 is the value
  of P(x) for xkn-1
• By receiving any k values of the polynomial, we
  can reconstruct the polynomial and calculate
  Y0Yk-1

0
k-1
19
Polynomial codes properties

• Optimal encoding for every (n,k), where n may not
  be defined in advance.
• Encoding and decoding using either
• Reed-Solomon
• Vandermonde matrix
• Reed-Solomon complexity (in field operations)
• Trivial encoding O(k2k(n-k))
• Trivial decoding O(k2e)
• There exist decoding in O(n log2n loglog n) with
  very large factor, making it unusable.

20
Randomized codes

• Codes in which the encoder uses randomization to
  encode the message.
• With high probability decoding of the message is
  reached after receiving (1e)k encodings.
• Example In polynomial codes, the indexes can be
  randomly picked from the range 0..k2-1.Due to
  the birthday paradox, no index will be used twice.

21
XOR based random codes

• A vector of v size k over GF(2) is randomly
  picked.
• The encoder send ltv,x1..xkgt The seed for
  generating the random vector.
• The decoder after receiving k independent
  vectors, may solve the proper linear equations
  and reconstruct x1..xk.

22
XOR based random codes (cont)

• The expected number of equations to reach the
  decoder from which k are independent is no more
  than k2
• The expected encoding complexity is O(k2)
• The expected decoding complexity is O(k3)
• We need to construct a more efficient method

23
Tornado Codes

• M. G. Luby,
• M. Mitzenmacher,
• M. A. Shokrollahi,
• D. A. Spielman,
• V. Stemann

24
Tornado codes

• Define a special bipartite graph between the
  original packets and redundant XOR packets.
• The received packets form linear equations, to
  which the solution is the message packets.
• With high probability solution is found.

25
Encoding Structure
Bipartite graph
Bipartite graph
Standard loss-resilient code
Message n to encoding cn, c2ß1-1/c is the
shrink factor
message
redundancy
26
Encoding Process
27
Decoding Process Direct Recovery
28
Decoding Process Substitution Recovery
indicates right node has one edge
29
Decoding Process Substitution Recovery
indicates right node has one edge
30
Tornado Complexity efficiency

• Efficiency relaxation
• Decode from any (1 e)n words of encoding
• Reception efficiency is 1/(1 e).
• Complexity (using XOR operations)
• Encoding O(n ln(1/e))
• Decoding O(n ln(1/e))

31
Tornado limitations

• The channel rate must be known in advance, as for
  each rate we need to construct a different
  bipartite graph.
• The bipartite graph construction must be known by
  both peers at the beginning.

32
Onlineness limitations

• A code is not online, if the reconstruction of
  lost packets is delayed until enough redundant
  packets are received.
• With all shown erasure techniques, decoding can
  not be performed online. For some losses, we
  should wait for at least k encodings in order to
  decode anything.

Need at least k encodings for decoding losses
lost packet
33
Bulk data distribution

• Data Carousel
• RMDP
• Digital Fountain

34
Model

• Single server
• Multiples receivers
• Single bulk data for distribution
• Software, Databases, Video
• Networking IP, Cellular, Satellite

35
TCP like solution

• Each client communicate with the server for a
  continuous-connection download.
• Server resources exhaustion.
• Bandwidth , Connections, Disk accesses, CPU
• Need to handle TCP disconnections.

36
Data carousel

• Receiver signup for a multicast group.
• Data is fragmented into packets which are
  transmitted in a loop.
• For loss probability P, the average rounds
  required for receiving all packets is expected
  as O(log1/p n)
• Using FEC with n slightly larger than k, does not
  help much.

37
RMDP Reliable Multicast data Distribution
Protocol

• L. Rizzo
• L. Vicisano

38
RMDP - Server

• Data is encoded with ngtgtk
• Each receiver request for data, reports the
  receivers downstream bandwidth.
• Server transmits encoded symbols at the maximal
  bandwidth of all receivers.
• Server keeps track of receivers expected
  reception. When a receiver receives enough
  packets, it is removed.

R
R
S
R
39
RMDP - Receivers

• Receivers accept enough distinct packets (k) in
  order to construct the original k data packets.
• The number of packets a receiver accepts on the
  average is its bandwidth multiplied by its
  reception time.
• After a receiver is assumed to accepts n
  messages, it may be considered finished.

40
RMDP
41
RMDP - Efficiency

• Encoding complexity is high O(nk) where n is
  much larger than k.
• Decoding complexity O(k2rk), where r is the
  number of packets received which are not part of
  the original data.
• Using other erasure techniques may improve
  efficiency.

42
Digital Fountain

• J. W. Byers
• M. G. Luby
• M. Mitzenmacher
• A. Rege

43
Digital Fountain

• Tornado-Codes used as erasure technique.
• Server transmit data over different layers, with
  increasing rate.
• Each layer is transmitted over a different
  multicast group.
• Example Each layer has about twice the
  transmission rate as the one beneath it.BW1BW0
  BWigt12BWi-12i-1BW0

44
Receivers

• Receiver which subscribes to layer i, actually
  subscribes to all multicast groups of layers
  0i-1
• With the example before,Reception bandwidth
  2BWi2iBW0
• If only a few losses were during some time, the
  receiver joins a higher layer.
• If more than about BWi losses were detected, the
  receiver retreat to a lower layer.

BW4
BW3
BW2
BW1
BW0
45
Congestion control

• Synchronization Points (SP) are special packets
  marking receivers of the time they may join
  higher levels.
• Servers periodically transmit with bursts of
  twice the rate of each layer.
• Receivers which experienced few losses are
  candidate to move to a higher layer.
• Thos who experienced many losses, should lower
  their layer.

46
Packets scheduling

• A receiver which signed-up to layer i should
  receiver as many distinct packets from all its
  layers.
• Moreover, a receiver which signed-up to lower
  layer, should accept all distinct packets from
  its layer.

47
Sample scheduling

• The following scheduling was proposed

48
Parallel downloading

• The model
• Single client, many servers
• Servers may go down dynamically
• Connectionless
• P2P Kazza, eMule,

49
Parallel downloading

• The encodes the message with k2ltltn?8.
• The indices may be randomly picked or be uniquely
  constructed using the computers IP.
• The decoder reconstructs the message from (1e)k
  distinct encodings.
• Simple and neat, but unimplemented.

50
LT Codes

• M. G. Luby

51
LT codes

• Rate-less codes, i.e. there is no predetermined
  ratio n/k. In LT codes, n2k which is
  practically infinite.
• Similar to Tornado-Codes.
• Encoding symbols are XORs of data symbols.
• Code is not systematic, meaning that data symbols
  are rarely part of the encoding.

52
LT codes description
Each encoding symbol is a XOR of some data
symbols.
The degree of the encoding symbols has a
predetermined distribution.
53
Encoding process

• While still need an encoding symbol
• Pick a degree d from the appropriate degree
  distribution
• Randomly pick d data symbols
• Encode them as encoded symbol by using XOR
  operations.

54
Decoding process

• Same as in Tornado Codes
• While there are new encoding symbols waiting or
  an encoding with degree 1 exists
• Find an encoding symbol with degree 1.
• Set its appropriate data symbol.
• Extract data from all neighboring encoding
  symbols.

55
Degree distribution
56
Degree distribution
57
LT codes efficiency

• The probability that the decoding process will
  succeed after decoding
  symbols is (1-d).
• The decoding of each symbol complexity is on
  average the average degree of encoding symbols
  O(ln(k/d))
• The efficiency is dictated by the encoding degree
  distribution.

58
Decoding rate in LT codes
59
Pros Cons

• Encoding Decoding complexity is not linear as
  in Tornado-Codes
• No predetermined size of encoding (rateless)
• No preprocessing for graph construction
• Extra encoded symbols required for successful
  decoding is asymptotically vanishing
• No differential equations in the analysis
• Not a Real-Time scheme, as symbols are decoded
  almost entirely by the last encoding received.

60
Real-Time Oblivious Erasure Correcting

• Amos Beimel
• Shlomi Dolev
• Noam Singer

61
Motivation

• Servers with large files
• Many downloading client
• Non reliable communication

62
Motivation (cont)

• Problem
• Channels with high loss rate.
• Expensive feed-back channels
• Current solutions
• ARQ Requires large feed-back
• FEC High decoding complexity
• Our innovation
• Meeting them half way

63
Background

• Previous works
• rate-less codes
• Real-time codes
• Our contribution

64
Previous works

• I.S.Reed, G.Solomon - 1960
• Optimal efficiency
• Quadratic complexity
• Tornado, LT, Raptor Codes / Luby, Mitzenmacher,
  Shokrollahi, Spielman, Stemann 1988? -2004
• Efficiency (1e)k
• Linear complexity
• Graph construction

65
Previous works (cont)

• LT Codes / M.G.Luby - 2002
• Efficiency
• Complexity O(k log k)
• Online Codes / Peter Maymounkov Raptor Codes /
  M. A. Shokrollahi - 2002
• Efficiency (1e)k
• Linear complexity

66
Rate-less Codes

• No predetermined channel rate Rk/n
• Can basically send 8 encodings
• Can decode the message from any large enough
  subset of symbols.
• Rate-less RS, LT Codes, Online Codes
• Non rate-less Tornado Codes

67
Real-Time Codes

• Complexity
• Symbol encoding
• Symbol reception
• Message decoding
• Decoding
• Rate in which a symbol is decoded
• Continuous decoded prefix
• CPU balance over the entire process.

68
Our contribution

• Combine ARQ with Erasure correcting
• Rate-Less
• Real-Time
• Low decoder memory overhead
• Reduced feed-back
• Efficiency O(k)
• Complexity O(k log k)

69
Our protocol

• Protocol description
• Protocol properties
• Decoding probabilities
• Handling special cases

70
Oblivious decoding

• Each encoding is XORed from some input symbols.
• Decoding is made if exactly one symbol is
  missing.
• Otherwise, encoding is dumped.

71
Decoding
Encodings
Data Symbols
72
Protocol description

• Calculate degree m
• Pick d symbols
• XOR them
• Transmit encoding

• Check if exactly 1 symbol missing
• If not, dump
• Otherwise decode
• Transmit revealed count

Encodings
d3
Feed-back
r3
73
Protocol properties

• Low memory overhead
• Unrevealed bitmap of size k
• Can be optimized to O(1) encodings.
• Trivial encoder/decoder
• Rate-Less
• Real-Time

74
Decoding probability P(d,r)

• Conventions
• d Encoding degree
• r Revealed/Decoded symbols count
• Decoding only if exactly one symbol is missing

75
P(d,r) samples

• P(d1,r0) 1
• P(dk,rk-1) 1
• P(d1,r) (k-r)/k
• P(d1,rk/2) P(d2,rk/2) 0.5
• P(dgtr1,r) 0

76
Decoding probability P(d,r)

• There are d possible missing symbols
• Each can be from the (k-r) missing symbols
• The rest of the (d-1) are from the remainingr
  out of (k-1), (r-1) out of (k-2),
• Or

77
P(d,r) for d1
78
P(d,r) for d2
79
P(d,r) for any d
80
P(d,r) maximization on d
81
Maximization using d

• For each r the optimal degree d, to which P(d,r)
  is maximized.
• P(d-1,r,k) P(d,r) P(d1,r)
• Result

82
Optimal degree d(r)
d2
d1

83
Degree changes

• For ?
• There are different values for d
  until
• Remaining values of r possible
  different values for d
• May achieve feed-back messages

84
Optimal probability

• Decoding probability for using optimal degree
• Lemma

85
Optimal probability gt1/e
86
Decoding rate

• After expected a decoding occur
• Expected less than e symbols
• Total expected required encodings

87
Required encodings (graph)
88
Decoding rate (graph)
encodings
5 experiments
89
Decoding Rate (LT Codes)
encodings
5 experiments
90
Expected required encodings

• exp Expected total symbols required
• exp e?k
• We will later show exp 2?k
• Experiments show expk?1.856

91
Proof (explt2k)

• Assume probability lt P(r)
• Experiments show exp 1.856k

92
The last symbols

• The degree d changes on each decoding.
• Feed-back delays reduces decoding efficiency
• Analysis show
• We may use the same degree of
• Expected required encodings

93
Conclusion - protocol properties

• Low memory overhead
• Missing bitmap of size k
• May be optimized to O(1) symbols.
• Trivial encoder/decoder
• Rate-Less
• Real-Time
• Low feed-back messages
• Robustness with feed-back losses
• Decoding rate gt 1/e
• Efficiency k?1.856
• Complexity O(k log k)

94
Heuristic enhancements

• First k encodings
• Estimating r

95
First k encodings

• Sender sends first symbols as is
• Another option
• Send a permutation
• May resolve loss bursts
• Efficient for higher channel rate R
• exp Rk(1-R)ke k(e-R(e-1))
• Non rate-less

96
Estimating d

• Feed-back messages are delayed or lost
• Inaccurate knowledge of r, reduces process
  efficiency.
• Server may estimate r using
• Assuming constant channel rate R
• Estimate r from the global decoding rate.
• Estimate r using a local adaptive estimation.

97
Application

• Smooth non-sequential transfer
• ARQ enhancement
• Broadcast over a tree structure
• Multi source download / immigration

98
Smooth non-sequential transfer

• Ordered transfer may not be required
• Data base items
• Pictures (non-compressed)
• MPEG
• Partial message may be enough
• Commercials
• News updates
• Decoding rate is at least 1/e

99
ARQ enhancement

• Used on retransmitting of a window
• Sending encodings instead of the entire window
• Feed-back messages
• Continuous decoded prefix
• Decoded symbols count on window
• May reduce feed-back messages
• Robustness with feed-back losses
• Useful with low channel rates

100
Broadcast over a tree structure

• Encodings sent from ancestors to children
• Feed-back (r) replied to ancestors
• Children cant know more than ancestors
• Each middle host forwards encodings or encodes
  from its decoded set
• Encoding indexes are different for each encoder.

101
Multi source download / immigration

• Client may communicate with several servers
• To each it send its r
• Each server generates independent encodings
• Servers crash/loss would not interrupt transfer

102
The End

• Thank you