DIGITAL COMMUNICATION

1
DIGITAL COMMUNICATION

• Error - Correction
• November 15, 2011
• A.J. Han Vinck

2
Error correction is needed in high tech devices
3
fundamental limits set by Shannon (1948)
The Bell System Technical Journal, Vol. 27, pp.
379423, 623656, October, 1948.
Bell Labs, 1955.
Fundamental problem reproducing at the receiver
a message selected at the transmitter Shannons
contribution - bound on the efficiency
(capacity) - how to achieve the bound
4
Overview of the lecture

• The position of channel coding in a communication
  chain
• Some channel models
• Burst error models
• Detection rules
• Intro to the coding problem
• performance
• The class of linear codes
• Bounds on the minimum distance
• Examples of codes
• Why error correction?

5
Overview of the lecture

• The position of channel coding in a communication
  chain
• Some channel models
• Burst error models
• Detection rules
• Intro to the coding problem
• performance
• The class of linear codes
• Bounds on the minimum distance
• Examples of codes
• Why error correction?

6
The communication model
k
data reduction/ compression
K
data protection
source
n
k
K
Message construction
decoder
sink
7
Point-to-point
transmitter
receiver
channel
physical
modem
modem
message bits
bits message
Signal generator
Signal processor
8
Position of Error Control Coding
signal generator
detector
channel
k input bits
k output bits
coding
signal generator
ECC
ECC
channel
detector
n input bits
k
k input bits
n
channel
coded signal generator
detector/decoder
k input bits
k output bits
coded modulation
9
transmission model (OSI)
Data Link Control
Data Link Control
Transmission of reliable packets
Physical
Physical
Unreliable trans-mission of bits
link
10
Something to think about
message bits
message compression protection of bits
Error correction
MPEG, JPEG, etc
bits message
correction of incorrect bits decompression
message
Compression reduces bit rate Protection increases
bit rate
11
Overview of the lecture

• The position of channel coding in a communication
  chain
• Some channel models
• Burst error models
• Detection rules
• Intro to the coding problem
• The class of linear codes
• Bounds on the minimum distance
• Examples of codes
• Why error correction?

12
Memoryless channel
Input X P(yx) output Y
transition probabilities
memoryless - output at time i depends only on
input at time i - input and output alphabet finite
13
Example binary symmetric channel (BSC)

• 1-p
• 0 0
• p
• 1
• 1-p

Error Source
E
X

Output
Input
E is the binary error sequence s.t. P(1)
1-P(0) p X is the binary information sequence Y
is the binary output sequence
14
from AWGN to BSC
Gaussian noise with prob. density function
Decide or -
X /- A
Y X N

p
Homework calculate the capacity as a function of
A and s2
15
Other models
1-e e e 1-e
0 1
0 (light on) 1 (light off)
0 1
0 E 1
X Y
p 1-p
P(X0) P0
P(X0) P0
Z-channel (optical)
Erasure channel (MAC)
16
the erasure channel
Application cdma detection, disk arrays
1-e e e 1-e
0 1
0 E 1
Disk 1
Disk 2
x y
Disk 3
Known position of error
Disk 4
Disk 5
P( x 0) 1 P( x 1 ) P0
17
From Gaussian to binary to erasure
e


xi /-
yi xi e

-
-
Output
Input
-



E
E
E
-
-
-

18
Erasure with errors
1-p-e
0 1
0 E 1
e
p
p
e
1-p-e
19
Overview of the lecture

• The position of channel coding in a communication
  chain
• Some channel models
• Burst error models
• Detection rules
• Intro to the coding problem
• performance
• The class of linear codes
• Bounds on the minimum distance
• Examples of codes
• Why error correction?

20
Modeling networking
packet
- 1-error causes retransmission - long packets
always have an error - short packets with ECC
give lower efficiency
Ack/Nack
Suppose that a packet arrives correctly with
probability Q. What is then the throughput as a
funtion of Q?
21
A Simple code

• For low packet loss rates (e.g. 5), sending
  duplicates is expensive (wastes bandwidth)
• XOR code
• XOR a group of data pkts together to produce
  repair pkt
• Transmit data XOR can recover 1 lost pkt

Å
Å
Å
10101
00111
11100
11000
10110

Å
Å
Å
10101
11100
00111
11000
10110
22
Density increases sensitivity
Blue Laser
CD
DVD
23
modeling
How to model scratches on a CD? Answer is
important for the design of ECC
24
Modeling binary transmission
test sequence
error sequence
channel
Problem 0 1 0 0 0 1 0 1 0 0 0 0 1 1 1
Determination of burst and guard space
burst
guard
burst
25
A simple error detection method
Fill row wise Transmit column wise RESULT any
burst of length L can be detected
0 1 1 0 0 1 1 0 0 1 1 0 0 0 1 1 0 0 1 1 0 0 0
0 0 0 0 0 0 0 0 0 0
L
0 1 1 0 0 1 1 0 0 0 0 0 0 0 1 1 0 0 1 1 1 1 0
0 0 0 0 0 0 0 0 1 1
row parity
What happens with bursts of length larger than L?
26
burst error model (Gilbert-Elliot)
Random error channel outputs independent
P(0) 1- P(1)
Error Source
Burst error channel outputs dependent
P(0 state bad ) P(1state bad )
1/2 P(0 state good ) 1 - P(1state good
) 0.999
Error Source
State info good or bad
transition probability
Pgb
Pbb
good
bad
Pgg
Pbg
27
question
Pgb
Pbb
good
bad
Pgg
Pbg
P(0 state bad ) P(1state bad )
1/2 P(0 state good ) 1 - P(1state good
) 0.99 What is the average for P(0) for Pgg
0.9, Pgb 0.1 Pbg 0.99, Pbb 0.01
? Indicate how you can you extend the model?
28
Fritchman model for mobile radio channels
multiple states G and only one state B Closer to
an actual real-world channel
29
Example (from Timo Korhonen, Helsinki)

• In fading channels received data can experience
  burst errors that destroy large number of
  consecutive bits. This is harmful for channel
  coding
• Interleaving distributes burst errors along data
  stream
• A problem of interleaving is introduced extra
  delay
• Example below shows block interleaving

Received interleaved data
1 0 0 0 1 1 1 0 1 0 1 1 1 0 0 0 1 1 0 0 1
1 0 0 0 1 1 1 0 1 0 1 1 1 0 0 0 1 1 0 0 1
Block deinterleaving
Recovered data
1 0 0 0 1 0 0 0 1 0 1 1 1 1 0 1 1 0 1 0 1
30
example

• Consider the code C 000, 111
• A burst error of length 3 can not be corrected.
• Let us use a block interleaver 3X3

2 errors
Interleaver
Deinterleaver
1 error
1 error
1 error
31
Middleton type of burst channel model
0 1
0 1
Transition probability P(0)
channel 1
channel 2
Select channel k with probability Q(k)

channel k has transition probability p(k)
32
Impulsive Noise Classification
(a) Single transient model

• Parameters of single transient
• peak amplitude - pseudo frequency f0 1/T0-
  damping factor- duration- Interarrival Time

Measurements carried out by France Telecom in a
house during 40 h 2 classes of pulses (on 1644
pulses) single transient and burst
33
Interleaving from bursty to random
bursty
Message interleaver channel
interleaver -1 message encoder
decoder
random error
Note interleaving brings encoding and decoding
delay Homework compare the block and
convolutional interleaving w.r.t. delay
34
Interleaving block
Channel models are difficult to derive - burst
definition ? - random and burst errors ? for
practical reasons convert burst into random error
read in row wise
transmit column wise
1 0 0 1 1
0 1 0 0 1
1 0 000
0 0 1 1 0
1 0 0 1 1
35
De-Interleaving block
read in column wise this row contains 1 error
1 0 0 1 1
0 1 0 0 1
1 e e e e
e e 1 1 0
1 0 0 1 1
read out row wise
36
Interleaving convolutional
input sequence 0 input sequence 1 delay of b
elements input sequence m-1 delay of
(m-1)b elements Example b 5, m 3
in
out
37
ExampleUMTS Turbo Encoder
Systematic Output Xk
Input Xk
Upper RSC Encoder
Uninterleaved Parity Zk
Output
Lower RSC Encoder
Interleaved Parity Zk
Interleaved Input Xk
Interleaver

• From ETSI TS 125 212 v3.4.0 (2000-09), UMTS
  Multiplexing and channel coding
• Data is segmented into blocks of L bits, where 40
  ? L ? 5114

38
UMTS InterleaverInserting Data into Matrix

• Data is fed row-wise into a R by C matrix. R 5,
  10, or 20, 8 ? C ? 256
• If L lt RC then matrix is padded with dummy
  characters.

Data is permuted within each row.
Data is read from matrix column-wise
Rows are permuted
39
Overview of the lecture

• The position of channel coding in a communication
  chain
• Some channel models
• Burst error models
• Detection rules
• Intro to the coding problem
• performance
• The class of linear codes
• Bounds on the minimum distance
• Examples of codes
• Why error correction?

40
A channel transition model
Input X channel transition Output Y
x1 y1 x2 y2 xM
yN
P(y1 x1)
P(y2 x1)
P(yN x1)
P(y1 xM)
P(yN xM)
41
Error probability (MAP)

• Suppose for a received vector y we assign the
  message i as being transmitted.
• Then, the probability of a correct decision P(
  xi transmitted y received )
• For the assignment j we have P(j is correcty)
  P( xj transmitted y rec.)
• Hence, in order to maximize the probability of
  being correct we assign the i
• to the received vector y that maximizes P( xi
  transmitted y received )
• (Maximum Aposteriori Probability, MAP)

42
Maximum Likelihood (ML) receiver
find i that maximizes P( xi y)
P( xi , y ) / P( y ) P( y xi ) P ( xi
) / P( y ) for equally likely xi this is
equivalent to find maximum P( y xi )
43
example
For p 0.1 and X1 ( 0 0 ) P( X1 1/3
) X2 ( 1 1 ) P( X2 2/3) Give your MAP
and ML decision for Y ( 0 1 )
44
Overview of the lecture

• The position of channel coding in a communication
  chain
• Some channel models
• Burst error models
• Detection rules
• Intro to the coding problem
• performance
• The class of linear codes
• Bounds on the minimum distance
• Examples of codes
• Why error correction?

45
Bit protection

• Obtained by Error Control Codes (ECC)
• Forward Error Correction (FEC)
• Error Detection and feedback (ARQ)
• Performance depends on error statistics!
• Error models are very important

46
example
Transmit 0 0 0 or 1 1 1 How many errors can
we correct? How many errors can we
detect? Transmit A 00000 B 01011 C
10101 D 11110 How many errors can we
correct? How many errors can we detect? What is
the difference?
47
Practical communication system design
Code book
Code word in
receive
message
estimate
2k
decoder
channel
Code book
with errors
n
There are 2k code words of length n k is the
number of information bits transmitted in n
channel uses
48
Channel capacity
Definition The rate R of a code is the ratio
k/n, where k is the number of information bits
transmitted in n channel uses Shannon showed
that for R ? Capacity an encoding methods
exist with decoding error probability 0
49
Encoding and decoding according to Shannon
Code 2k binary codewords where p(0) P(1)
½ Channel errors P(0 ?1) P(1 ? 0) p i.e.
? error sequences ? 2nh(p) Decoder search
around received sequence for codeword with ? np
differences
space of 2n binary sequences
50
decoding error probability

• For t errors t/n-pgt ?
• 0 for n ? ?
• (law of large numbers)
• 2. gt 1 code word in region
• (codewords random)

51
A pictorial view

• 2n vectors 2k code words
•  

52
decoder

• Compare received word with all possible codewords

code words received  

• Decode the code word with minimum of
  differences (Most Likely)

53
example

• code words 0 0 0 0 0 0 1 0 1 1 1 0 1 0 1 1
  1 1 1 0
• received 0 0 0 1 1
• difference 0 0 0 1 1 0 1 0 0 0 1 0 1 1 0
  1 1 1 0 1
• best guess 0 1 0 1 1

only 1 difference
54
we have some problems

• Mapping from information to code words
• generation of code words (mutually far apart)
• storing of code book (2k code words, length n)
• Decoding
• Compare a received word with all possible code
  words

55
What are the classical research problems
Codebook maximize - of codewords of length
n - minimum
distance dmin

• Channel characterization
• Types of errors memory in the noise behavior ?
• Channel Capacity?

• Decoder
• Design of a decoding/decision algorithm
• Minimize complexity

56
Overview of the lecture

• The position of channel coding in a communication
  chain
• Some channel models
• Burst error models
• Detection rules
• Intro to the coding problem
• performance
• The class of linear codes
• Bounds on the minimum distance
• Examples of codes
• Why error correction?

57
Definitions

• Hamming distance between x and y is
• dH d(x, y) is the of positions where xi
  ¹yi
• The minimum distance of a code C is
• dmin min d(x, y) x Î C, y Î C, x y
• Hamming weight of a vector x is
• - w(x) d(x, 0 ) is the of positions where
  xi ¹0

58
example

• Hamming distance d( 1001, 0111) 3
• Minimum distance (101, 011, 110) 2
• Hamming weight w(0110101) 4
• Hamming was a famous scientist from Bell-lab and
  inventer of the Hamming code.

59
Performance

• A code with minimum distance dmin is capable of
  correcting t errors if
•  
• dmin ³ 2 t 1.
• Proof If t errors occur, then since dmin ³
  2 t 1
• an incorrect code word has at least t1
  differences with the received word.
•  

60
picture
A
B
2t1 differences
t differences from B
t differences from A
61
Performance

• A code with minimum distance dmin is capable of
  correcting E erasures if
•  
• dmin gt E.
• Proof
• If E lt dmin erasures occur, then at least 1
  position
• is left to distinguish between any two codewords.
• Note an erasure is a position where the receiver
  knows that an error occured.
•  

62
Performance

• A code with minimum distance dmin is capable of
  correcting E erasures
• and t errors if
•  
• dmin gt 2t E.
• Proof
• The minimum distance is reduced at maximum by the
  value E.
• Hence, if
• dmin - E gt 2t
•  we can still correct t errors

63
Performance for the Gausian channel


pE
p3
E
p3
pE
-
-
-

p1
p3
p2
P(error)
P(erasure)

• For an error correcting code 2t 2 n (p2 p3 )
  error correcting
• With erasures dmin gt n (p2 p1 ) 2 n p3
• Since p2 lt p1 error correction without
  erasures is always better!

64
Overview of the lecture

• The position of channel coding in a communication
  chain
• Some channel models
• Burst error models
• Detection rules
• Intro to the coding problem
• performance
• The class of linear codes
• Bounds on the minimum distance
• Examples of codes
• Why error correction?

65
LINEAR CODES

• Binary codes are called linear iff
•  
•    the component wise modulo-2 sum of two code
  words is again a code word.
•  
• Consequently, the all zero word is a code word.

66
LINEAR CODE GENERATOR

• The code words are
• - linear combinations of the rows of a binary
  generator matrix G with dimensions k, n
• - G must have rank k!
• Example Consider k 3, n 6. 
• 1 0 0 1 1 0
• generator matrix G 1 1 0 0 1 1
• 1 0 1 1 0 1
• (1,0,1)G ( 0, 0, 1, 0, 1, 1)

67
Systematic codes

• Let in general the matrix G be written as
•  
• 1 0 0 1 1 0
• G Ik P G 0 1 0 1 0 1 0 0 1 0
  1 1
• k 3, n 6
• The code generated is
• linear, systematic
• has minimum distance 3.
• the efficiency of the code is 3/6.

68
Example (optimum)

• Single Parity check code dmin 2, k n-1

100 0 1 0100 0
1 G In-1 P
00 01 1 All codewords have even weight!
69
Example (optimum)

• Repetition code dmin n, k 1
• G 1 1 1

70
Equivalent codes

• Any linear code generator can be brought in
  systematic form
• Gsys
• k
• n
• k
  n
• 
  
• n
• Note the elementary operation have an inverse.
• Homework give an example for k 4 and n 7

Elementary row operations
Non-systematic form
Elementary column operations
71
Property

• The set of distances
• from all code words to the all zero code word
• is the same as to any other code word.
• Proof
• d( x, y ) d( x Å x, z y Å x ) d( 0, z ),
• by linearity z is also a code word.

72
Thus!

• the determination of the minimum distance of a
  code is equivalent to
•  
• the determination of the minimum Hamming weight
  of the code words.
•  
• The complexity of this operation is proportional
  to of code words
•  

73
example

• Consider the code words
• 00000
• 01101
• 10011
• 11110
• Homework Determine the minimum distance

74
Linear code generator

• I(X) represents the k bit info vector ( i0, i1,
  ..., ik-1 )
• g(X) is a binary polynomial of degree ( n-k )
• THEN
• the code vector C of length n can be described
  by
•  
• C(X) I(X) g(X) all operations modulo-2.

75
EX k 4, n 7 and g(X) 1 X X3

• For the information vector (1,0,1,0)
•  
• C(X) (1 X2 ) ( 1 X X3 )
• 1 X X2 X5 º (1,1,1, 0, 0,1, 0).
•  the encoding procedure in (k x n) matrix form
• 1 1 0 1 0 0 0
• G 0 1 1 0 1 0 0 c I G
• 0 0 1 1 0 1 0
• 0 0 0 1 1 0 1

76
Implementation with a shift-register
The following shift register can be used g(X)
(1 X X3 )
ik-1 ... i2 i1 i0
Homework give a description of the shift
control to obtain the result
77
Some remarks

• Generators for different k and n
• are constructed using mathematics
• listed in many text books
• What remains is the decoding!

78
Overview of the lecture

• The position of channel coding in a communication
  chain
• Some channel models
• Burst error models
• Detection rules
• Intro to the coding problem
• performance
• The class of linear codes
• Bounds on the minimum distance
• Examples of codes
• Why error correction?

79
Bounds on minimum distance (Hamming)

• Linear codes have a systematic equivalent G
• Minimum Hamming weight n k 1
• (Singleton bound)
• code words correctable error patterns 2n
• Homework show that Hamming codes satisfy the
  bound with equality!

80
Hamming bound example

• Problem
• Give the upper bound of the size of a linear
  code C of length n 6 and dmin 3
• Solution
• the code corrects up to one error and thus C
  26 / ( 1 6) 64 / 7 C 9
• Since for a linear code C 2k we have C
  8
• example 1 0 0 0 1 1
• G 0 1 0 1 0 1
• 0 0 1 1 1 0

81
Bounds on minimum distance (Plotkin)
- List all codewords of a binary linar code
n 2k
codewords - Every column must have 2k/2 ones
(proof) (no all zero column exists)
Conclusion The total number of ones in the
codewords is n x 2k/2 gt dmin must be ?
average, and thus dmin ? n x 2k-1/ (2k-1)
82
Bounds on minimum distance (Gilbert)

• Start Select codeword from 2n possible words
• 1. Remove all words at distance lt dmin from
  selected codeword
• 2. Select one of the remaining as next codeword
• 3. Goto 1. unless no possibilities left.
• RESULT
• homework show that logM/n 1 h(2p) for dmin
  -1 2t _at_ 2pn p lt ¼

83
plot
1
Plotkin
Singleton
R log2M/n
1-h(p)
1-h(2p)
0
0.5
p _at_ t/n
84
For Additive White Gaussian Noise
Channelsquantized as binary symmetric channel

• Error probability p e-Es/No
• where Es is energy per transmitted symbol
• No the one-sided noise power spectral density
• For an uncoded system p e-Eb/No
• For a coded system with minimum distance d
• decoding errors occur when by 2t1 gt d
• nEs kEb and thus
• pc e-d/2 k/n Eb/No
• CONCLUSION make the factor C d/2 k/n gt 1

85
Overview of the lecture

• The position of channel coding in a communication
  chain
• Some channel models
• Burst error models
• Detection rules
• Intro to the coding problem
• performance
• The class of linear codes
• Bounds on the minimum distance
• Examples of codes
• Why error correction?

86
Richard Hamming (1915-1998)
x1, x2, ..., x7 are binary values (0 or 1) x1,
x2, x3, x4 are information bits x5 makes
x1x3x4x5 even x6 makes x2x3x4x6 even x7
makes x1x3x2x7 even x1, x2, ..., x7 are
transmitted (stored) Any single bit change can be
corrected
x4
x5
x6
x3
x1
x2
x7
In 1947, Hamming was one of the earliest users of
primitive computers at Bell Laboratories. He was
frustrated by their lack of fundamental
reliability. He therefore puzzled over the
problem of how a computer might check and correct
its own results. Within several months Hamming
discovered that extra bits could be added to the
internal binary numbers of the computer to
redundantly encode numerical quantities. This
redundancy enabled relatively simple circuitry to
identify and correct any single bit that was bad
within the encoded block of bits (typically one
word of data). This encoding scheme, now known as
Hamming Code, also detects the condition of any
two bits in the encoded block that fail
simultaneously. In 1950, Hamming published his
work as Error Detecting and Error Correcting
Codes' in the Bell System Technical Journal, vol.
29, pp. 147-160.
87
Simpler code, same principle
info check 1 0 0 ? ?
? x1 x2 x3 x4 x5 x6
x6
x2
x4
info check 1 0 0 0 1
1 transmit x1 x2 x3 x4 x5 x6
x3
x1
Decode received 1 0 0 0 0 1
x5
Q Transmission efficiency?
88
The next code has length 15 and 11 information
bits
P3
x11
p4
x6
x7
x4
x5
x3
x9
x2
x10
x1
x8
P1
P2
Q Check that 1 error can be corrected Q
Transmission efficiency?
89
Why making things complicated?

• The (3,1) code

P1
P2
X1
90

• Can we do a general presentation ? YES
  

(31,26)
91
Hamming codes

• Minimum distance 3
• Construction
• G Ik All m-tuples of Hamming weight gt 1
• where k 2m m - 1
• Check that the minimum distance is 3!
• Give the efficiency of the code

92
Example k 4, n 7, m 3

• 1000 110
• 0100 101
• G
• 0010 011
• 0001 111
• In Bluetooth we have a shortened (10,15) Hamming
  Code

93
Syndrome decoding
Let G Ik P then construct HT P
In-k For all code words c xG,
cHT xGHT 0 Hence, for a received noisy
vector ( c Å n ) HT c HT Å n HT n
HT S
94
example
110 101 011 100 010 001
100 110 G 010 101 001 011
x 1 0 1 c 1 0 1 1 0 1 c HT 0 0 0 n 0 1 0
0 0 0 c Å n 1 1 1 1 0 1 c Å n HT S 1 0 1
HT
Obvious fast decoder precalculate all syndromes
at receiver for correctable errors
95
In system form
c Å n
c Å n
Calculate syndrome
c Å n Å n
n
when n n then n Å n 0
Precalculated syndromes
c Å n HT S
Homework choose parameters that can be
implemented
96
The parity check matrix

• We saw that C I G G is k x n
• IGHT 0 HT is n x n-k
• Proposition Only for codewords CHT 0
• Proof take the systematic encoder presentation
• - there are 2n-k different syndromes
• - 2k different vectors cause the same syndrome
• nHT S mHT S (n Å m)HT 0 i.e. they
  differ by a codeword
• - the 2k codewords give the syndrome 0

97
The parity check matrix

• A property to be used later
• Any dmin-1 rows of HT are linearly independent
• Reason If not, then less than dmin rows give the
  syndrome 0
• Only Codewords give syndrome 0 and have
  minimum weight ³ dmin

98
Varshamov-Bound

• Let us construct a matrix HT with dimensions
  n x n-k for a given minimum distance d.
• Start with the identity matrix with dimension
  (n-k) x (n-k).
• Then, construct a list of vectors such that any
  d-2 vectors are linearly independent
• If at list size i,
• we can add one more vector, not equal to the all
  zero vector, different from the linear
  combinations and thus all d-1 combinations are
  linearly independent.
• Define n i1. Take the largest value of n such
  that
• thus,

99
The parity check matrix

• property
• for a code with minimum distance dmin 2t1,
• all error events of weight t give rise to a
  different syndrome.
• Reason
• If not, then the sum of two events with
  sum-weight less than dmin give the syndrome 0.
• This contradicts the assumption that only
  codewords give syndrome 0 and
• have minimum weight ³ dmin 2t1

100
Reed Solomon Codes (CD, DVD)

• Structure

m
k information symbols
n-k check symbols
Properties minimum distance n-k1
(symbols) length 2m -1
101
General remarks on RS codes

• The general problem is the decoding
• RS codes can be decoded using
• Euclids algorithm
• Berlekamp Massey algorithm

102
Reed-Muller based codes (UMTS)

• Starting code m 1 C 00,01,10,11
• has minimum distance 2m-1
• 2m1 code words U of length 2m
• NEW code with 2m2 code words of length 2m1
• (U,U) and (U,U)
• Þ distance (U,U), (U,U) 2m why?
  Convince yourself!
• Þ distance (U,U), (V,V) 2 2m-1 2m
• Þ distance (U,U), (V,V) 2 2m-1 2m
  (use compl.prop.)

103
Reed-Muller based codes (UMTS)

• Suppose we have a generator G for certain m
• The construction of G for m1 is simple
• Example
• m 1 m2 m
  3 etc.
• G 01 G 0 1 0 1 0 1 0 1 0 1 0 1
• 11 1 1 1 1 1 1 1 1 1
  1 1 1
• 0 0 1 1 0 0 1 1 0 0 1 1
• 0 0 0 0 1 1 1 1

• dmin 1 2 4
  

104
Reed-Muller based codes (UMTS)

• Basis for construction
• Take 2 linear code generators G1 and G2
• of length n
• and minimum distance D1 and D2, respectively
• Then, G1 G1
• G has dmin min2D1,D2
• 0 G2
• Proof!

105
ISBN numbering code

• for the ISBN numbers (a1, a2, , a10) we have
• where
• We use a special symbol X in case a10 10.
• A single transmission error gives as a result
• since 11 is a prime number.
• Note check that the code can also detect a
  double transposition (interchange of 2 numbers).

106
Varshamov-Tenengolz codes

• for the binary numbers (x1, x2, , xn) we have
• A single transmission error gives as a result
• Check that all numbers are different and thus we
  can correct 1 error!
• The cardinality of the code
• Homework compare the rate of the code with the
  rate of the Hamming code.

107
Overview of the lecture

• The position of channel coding in a communication
  chain
• Some channel models
• Burst error models
• Detection rules
• Intro to the coding problem
• performance
• The class of linear codes
• Bounds on the minimum distance
• Examples of codes
• Why error correction?

108
Why error correction?

• Systems with errors can be made almost error free
• CD, DvD would not work without RS codes

109
A multi user application

• ENCODING two users transmit 1 or 1
• - Suppose one user uses an error correcting code
  at rate ½.
• - The second user just sends his information at
  rate 1.
• CHANNEL adds the values and thus the receiver
  observes 2, 0 or -2
• DECODER decodes the information for user 1 using
  an erasure decoding procedure After decoding,
  the information from user 2 can be calculated
• SUM RATE 1.5 bit/transmission
• NOTE Time Division gives a rate 1!

110
A multi user application
User 1 R 1/2
/- 1
/- 2 0
add
User 2 R 1
/- 1
ERASURE Channel from user 1 to receiver
R2
½
1
2
1
Sum efficiency time sharing
½
0
½
½
R1
-1
-2
½
1
½
111
Why error correction?
112
Why error correction?

• Suppose
• transmit (n-k
  bits) receiver
• Y X Å N(oise) YHT Å
  NHT N Å X Y
• add XHT add X
• X X
  X X
  
• transmit n bits
• For n-k nh(p) we transmitted n nh(p) bits
  instead of 2n
• Reference Slepian and Wolf

113
Why error correction?

• In ARQ systems systems collaps can be postponed!

100 k/n
troughput
0
1
Channel error probability
114
Combining error detection-correction

• G1(X) G2(X) generates a code with minimum
  distance D2
• G1(X) generates a code with minimum distance D1 lt
  D2
• C(X) I(X) G1(X) G2(X) I(X)G1(X)
• decoding
• step 1 correct a maximum of ë( D1 1 )/2û
  errors with G1(X)
• step 2 detect the remaining errors with
  G1(X)G2(X)
• properties
• 1. t ë( D1 1 )/2û errors correctable
• 2. t D2 - 1 - ë( D1 1 )/2û errors
  detectable
• Homework construct some examples

115
Combining error detection-correction

• Example
• n k dmin generator polynomial (octal)
• 63 57 3 G1 103 (octal)
• 51 5 G2 x G1 G1 x (127)
• 45 7 G3 x G2 x G1 G2 x G1 x (147)

116
Example bluetooth

• Three error correction schemes are defined for
  Bluetooth
• 1/3 rate FEC, a simple repetition
• 2/3 rate FEC, a shortened Hamming Code (10,15)
• ARQ scheme for data (automatic retransmission
  request)
• FEC schemes reduce the number of retransmissions
• A CRC code decides whether a packet/header
  contains errors, i.e.
• transmit C(X) I(X) g(X)
• receive R(X) C(X) E(X)
• check R(X) modulo g(X) 0 ?

117
Code shortening

• Suppose that we have a RS code in systematic form
• n
• 1 0 ... 0
• 0 1 ... 0 The
  generator takes the first k n-k columns
• Parity part minimum distance is
  n-k1
• ...
• 0 0 ... 1
• k n-k
• k n-k
• take the last k rows of length k
  n - k
• 100...0 Parity part minimum
  distance is n k 1
• 00... 1

Case a
Case b
118
Channel with insertions and deletions

• Bad synchronization or clock recovery at
  receiver
• insertion
• 1 0 0 0 1 1 1 0 0 1 0 1 0 0 1 0 1
  1 1 0 0 1 0
• deletion
• 1 0 0 0 1 1 1 0 0 1 0 1 0 0 1 1 1 0
  0 1 0
• Problem finding start and end of messages

119
Channel with insertions and deletions

• Example
• the following code corrects
• a single inversion error
• or a single deletion/insertion error
• C 0 0 0 0 0 0, 1 1 1 1 1 1, 0 0 1 0 1 1, 1 1
  0 1 0 0
• dmin 3
• An insertion/deletion makes the word
  longer/shorter

120
Channel with insertions and deletions

• Due to errors in bit pattern
• flag 1 1 1 1 1 0, avoid 1 1 1 1 1 in
  frame
• 0 1 1 1 1 1 0 0 1 1 0 1 0 1 1 1 1
  0 1 0 0 1 1 0 1
• 0 1 1 0 1 0 1 0 0 1 1 0 1 insertion
• 0 1 1 1 0 0 0 0 1 1 0 1 0 1 1 1 1
  0 0 0 1 1 0 1
• 0 1 1 1 1 0 0 1 1 0 1 deletion

121
Channels with interference

• Example (optical channel)

Error probability depends on symbols in
neighboring slots
122
Channels with memory (ex recording)

• Example Yi Xi Xi-1 Xi Î 1, -1

Xi
Xi-1
Yi Î 2, 0, -2
123
The password problem
server
Password Honary
Calculate Hash(password)
Hash(Honary)
compare
I am Bahram
memory
Hash(Honary)
server
biometric
Calculate Hash( )
Hash( )
compare
I am Bahram
memory
Hash( )
124
Problem in authentication passwords need to
be exact biometrics are only very similar
125
The problem A secure sketch
enroll
sketch
B
Store BHT (syndrome)
calculate
B B Å N
BHT
BHT
BHT
(B Å B)HT NHT
reconstruction
B
N
DECODE N
From NHT 2k possible vectors B exp. correct B
2k x B/2n prob. correct guess
2n-k / B ³ 2-k
126
Authentication secure storage, with errors
random R
E(R)
R
Enrollment
Condition given S and Hash(R) it is hard to
estimate B, R
B
S B Å E(R)
store S
Å
Hash(R)
B
S Å B B Å E(R) Å B E(R) Å N gt R

gt Hash(R)
Hash(R)
authentication
S
Hash(R)
127
Authentication secure storage, with errors
random R
E(R)
R
Enrollment
Condition given S and Hash(R) it is hard to
estimate B, R
B
S B Å E(R)
store S
Å
Hash(R)
attack
B S Å E(R) B Å E(R) Å E(R) or S HT B Å
E(R) HT B HT
Guess R or B
S
Hash(R)
128
Binary entropy
interpretation let a binary sequence contain
pn ones, then we can specify each sequence with
log2 2nh(p) n h(p) bits
Homework Prove the approximation using ln N! N
lnN for N large. Use also logax y ? logb
x y logba The Stirling approximation ?
129
The Binary Entropy h(p) -plog2p (1-p)
log2 (1-p)
Note h(p) h(1-p)