Shashi Borade, Baris Nakiboglu, Lizhong Zheng

1
Unequal Error Protection Fundamental Limits and
Strategies
Shashi Borade, Baris Nakiboglu, Lizhong
Zheng EECS, MIT ITA 2008
Many thanks Bob Gallager, David Forney, Emre
Telatar, Greg Wornell
TexPoint fonts used in EMF. Read the TexPoint
manual before you delete this box. AA
2
Classical Theoretical Framework

• Semantic aspects of communication are irrelevant
  to the engineering problem.
• - Shannon, 1948.

3
Classical Theoretical Framework

• Semantic aspects of communication are irrelevant
  to the engineering problem.
• - Shannon, 1948.
• Trivializes meaning of information all
  information equally important.
• --- All mistakes equally costly
• --- reliability error probability over all
  messages
• Great insights in large bandwidth or delay limit
• Universal interface of bits any source over any
  channel
• A homogeneous view of information

4
Departure from classical framework

• When sufficient error protection is luxury
• Protecting all information equally either
  infeasible or inefficient
• Instead, protect a crucial part better

5
Departure from classical framework

• When sufficient error protection is luxury
• Protecting all information equally either
  infeasible or inefficient
• Instead, protect a crucial part better
• Dynamic wireless networks
• --- power control, frequency allocations
  more important than actual data.
• Internet packet header vs. payload data.
  Protect headers better in physical layer.
• Audio/video broadcast coarse vs. finer
  resolution
• Control systems critical vs. finer actions
• Heterogeneous nature of information

6
Previous work on UEP

• Simplest approach separate channels for
  different types of data
• Control channel in wireless short codes, low
  spectral efficiency
• Gray codes label QAM points to minimize bit
  errors
• Brief history of UEP (incomplete IT centric)
• Weighted PCM Bedrosian58 Bellman-Kalaba58
• Linear codes for UEP Masnick-Wolf67

7
Previous work on UEP

• Simplest approach separate channels for
  different types of data
• Control channel in wireless short codes, low
  spectral efficiency
• Gray codes label QAM points to minimize bit
  errors
• Brief history of UEP (incomplete IT centric)
• Weighted PCM Bedrosian58 Bellman-Kalaba58
• Linear codes for UEP Masnick-Wolf67
• Multilevel codes Calderbank-Seshadri93
• Priority Encoded Transmission Albanese et
  al96
• Diversity embedded codes Diggavi-Tse04
• many smart designs in communications, video,
  computer systems

8
Previous work on UEP

• Simplest approach separate channels for
  different types of data
• Control channel in wireless short codes, low
  spectral efficiency
• Gray codes label QAM points to minimize bit
  errors
• Brief history of UEP (incomplete)
• Weighted PCM Bedrosian58 Bellman-Kalaba58
• Linear codes for UEP Masnick-Wolf67
• Multilevel codes Calderbank-Seshadri93
• Priority Encoded Transmission Albanese et
  al96
• Diversity embedded codes Diggavi-Tse04
• many smart designs in communications, video,
  computer systems

9
A new notion Message-wise UEP
bits
messages
10
A new notion Message-wise UEP
bits
messages

• Existing UEP notion (bit-wise UEP)
• some bits have higher priority (say is
  special)
• higher priority better protection (packet
  headers, audio/video)

11
A new notion Message-wise UEP
bits
messages

• Existing UEP notion (bit-wise UEP)
• some bits have higher priority (say is
  special)
• higher priority better protection (packet
  headers, audio/video)
• Alternatively, some messages have higher priority
  (say is special)
• -- minimize
  (conditional error probability)
• -- crucial message too costly to miss. (system
  emergency)

12
A new notion Message-wise UEP
bits
messages

• Existing UEP notion (bit-wise UEP)
• some bits have higher priority (say is
  special)
• higher priority better protection (packet
  headers, audio/video)
• Alternatively, some messages have higher priority
  (say is special)
• -- minimize
  (conditional error probability)
• -- crucial message too costly to miss. (system
  emergency)
• Message error not related to any particular bit
• Significantly different than bit-wise

13
Homogenous vs. Heterogeneous

• Classically, bit errors and message errors are
  treated equally
• UEP better protection to selected parts of
  information
• These parts need not be separate bits
• Message-wise UEP an alternate way of
  differentiating parts of information

14
Talk outline

• Data-rate is critical resource at capacity
• Overall error probability decays very slowly
  with block-length
• Can at least a few special bits or special
  messages get better protection?
• Is there a general benchmark for UEP
  performance?
• -- No previously known fundamental limits

15
Talk outline

• Data-rate is critical resource at capacity
• Overall error probability decays very slowly
  with block-length
• Can at least a few special bits or special
  messages get better protection?
• Is there a general benchmark for UEP
  performance?
• -- No previously known fundamental limits

• Error exponents as performance benchmarks
• Bit-wise UEP
• -- Single special bit
• Message-wise UEP
• -- Single special message
• -- Many special messages
• First , no-feedback Later, full-feedback.

16
Achieving capacity

• General DMC capacity , transition matrix
• Length code, rate (i.e.
  messages) , error probability
• Classical exponent

17
Achieving capacity

• General DMC capacity , transition matrix
• Length code, rate (i.e.
  messages) , error probability
• Classical exponent

Single special bit

• Definition is best exponent when
  communicating reliably at
• 
  .
• Not even clear if

18
Theorem

• Geometric Interpretation
• Dotted lines denote decoding regions
• Output space split in two halves by each
  half has messages

19
Theorem

• Geometric Interpretation
• Dotted lines denote decoding regions
• Output space split in two halves by each
  half has messages
• Thick empty patch around equator

Impossible!
20
Single special message ( )

• Definition is best exponent when
  communicating reliably at
• 
  .

Theorem where is capacity achieving output
distribution
Compare to classical case no exponent if all
messages protected equally
21
Single special message ( )

• Definition is best exponent when
  communicating reliably at
• 
  .

Theorem where is capacity achieving output
distribution
Compare to classical case no exponent if all
messages protected equally

• Definition pseudo-capacity
  
  and optimal input.
• Shannon capacity
• -- Very noisy channels
• -- Symmetric channels like BSC

22

• Geometric Interpretation
• Optimal strategy
• Encoder Special message
• Decoder based on output empirical distribution

How large can one decoding region be? (while
filling small regions)
23
Many special messages

• First messages special (out of total
  messages)
• Definition is best exponent when
  communicating reliably at
• each special message
  .
• If only special messages achieve classical error
  exponent

24
Many special messages

• First messages special (out of total
  messages)
• Definition is best exponent when
  communicating reliably at
• each special message
  .
• If only special messages achieve classical error
  exponent
• With additional ordinary messages,

Theorem
25
Geometric intuition
output space
empty box

26
Geometric intuition
large balls of sphere-packing radius
box full of boulders

27
Geometric intuition
additional small regions
adding sand

28
Geometric intuition
additional small regions
adding sand
Two-stage Decoder First stage chooses class
special or ordinary -- blue region
or green Second stage ML within chosen
class

29
Full feedback case

• No need of fixed decoding time. could be
  random.
• Feedback code at , average decoding time

30
Full feedback case

• No need of fixed decoding time. could be
  random.
• Feedback code at , average decoding time

Single special bit

• Definition is best exponent when
  communicating reliably at

• Equals single-message
• exponent

Theorem
31
Optimal strategy Protects special bit using a
special message -- a buzzer indicating bit
error Feedback connects bit-wise and
message-wise UEP
send .
If correct, send other bits. Else, buzzer
message
Decoder If buzzer detected, declare erasure,
Repeat afresh . Else, ML decoding.
Missed buzzer bit error
32
Optimal strategy Protects special bit using a
special message -- a buzzer indicating bit
error Feedback connects bit-wise UEP and
message-wise UEP

If correct, send other bits. Else, buzzer
message
Decoder If buzzer detected, declare erasure,
Repeat afresh . Else, ML decoding.
Missed buzzer bit error
Many special bits
rate
rate
rate vs. reliability Simple linear tradeoff
33
Many priority layers
Successive refinability Each layers exponent
as if all lower levels were ordinary
exponent
.
rate
rate
rate
rate
rate
Onion peeling strategy Encoder If recent layer
decoded right, send next layer at , else
start buzzer Decoder After decoding each
layer, check is buzzer sent later If buzzer
detected, declare erasure. Repeat afresh.
Else, proceed to next layer.
34
Single special message ( )

• Definition is best exponent when
  communicating reliably at
• 
  .

Theorem

• Feedback does not increase pseudo-capacity

Feedback connects bit-wise UEP and message-wise
UEP
35
Avoiding false-alarms

• Definition is best exponent when
  communicating reliably at
• 
  .
• By Jensens inequality

Theorem where achieves capacity. Let
denote an optimal input.
Comparison to classical case if all messages
want equal good false alarm exponent it
cannot be larger than
36
Avoiding false-alarms

• Definition is best exponent when
  communicating reliably at
• 
  .
• By Jensens inequality

Theorem where achieves capacity. Let
denote an optimal input.
Comparison to classical case if all messages
want equal good false alarm exponent it
cannot be larger than
Optimal strategy Encoder Special message
Decoder Output type ,
All other types ML ordinary
message Reverse of the strategy for
37
Geometric Interpretation
How small and far can special decoding region
be? (from remaining small regions)
38
Geometric Interpretation
How small and far can special decoding region
be? (from remaining small regions)

• With full-feedback

Combine strategy for single special message
with Yamamato-Itoh strategy. (also achieves
)
Theorem
With feedback, unchanged but
improves.
39
Summary and Future directions

• A general framework for UEP
• Message-wise UEP and bit-wise UEP
• Pseudo-capacity
• Role of feedback

40
Summary and Future directions

• Active use of UEP
• Two-way channels, relay, broadcast distributed
  coordination, .
• Network optimization
• Efficient coding
• List and erasure codes
• Algebraic and LDPC codes
• Rates below capacity (preprint upcoming)
• Heterogeneous error protection new notions of
  UEP
• Avoiding false alarms