LowDelay Distributed Source Coding: Bounds and Performance of Practical Codes

1
Low-Delay Distributed Source Coding Bounds and
Performance of Practical Codes

• Ozgun Bursalioglu Ertem Tuncel

Allerton 2005
2
Outline

• Scalar coding of distributed sources
• From quantizers to graphs
• Zero-error entropy coding for bipartite graphs
• Comparison (for jointly Gaussian sources) of
• a graph-theoretic bound applied to
  high-resolution uniform quantization indices
• Slepian-Wolf limit applied to the same
• Oohamas outer bound

3
Outline

• A simple coding strategy based on bipartite graph
  coloring
• Comparison of achieved rates to the bounds
• Performance of non-uniform quantization
• Why compandors (compressor expandor) do not
  help
• Power of expansors (expandor compressor) for
  jointly Gaussian sources

4
Lossy DSC Problem (Berger Tung)
X N
Encoder
Decoder
Y N
Encoder
5
Low-Delay DSC
6
Low-Delay DSC
Scalar Encoder
Xt
QX

Scalar Decoder
Scalar Encoder

Yt
QY
7
From Quantizers to Graphs
8
From Quantizers to Graphs
9
Zero-Error Entropy Coding for

• Simplest strategy bipartite graph coloring
  followed by (separate) entropy coding of colors.
• Instantaneous coding (Yan Berger 00, Zhao
  Effros 02, Koulgi et al. 03) does not
  necessarily involve coloring.
• Uniquely decodable coding (Koulgi et al. 03)
  the largest possible set of valid codes.
• Asymptotic performance of all are the same.

10
Coloring versus non-Coloring

• Coloring-based entropy coding

000101 ? ? ?
000101 ? ? ?
10110101110 ? ? ?
10110101110 ? ? ?

• Instantaneous codes
• Each edge pair prefix-free at one end at least

0001011 ? ? ?
10110111 ? ? ?
11
Outer Bound on Zero-Error Codes

• Koulgi et al. (2003) proved that

12
Outline

• Scalar coding of distributed sources
• From quantizers to graphs
• Zero-error entropy coding for bipartite graphs
• Comparison (for jointly Gaussian sources) of
• a graph-theoretic bound applied to
  high-resolution uniform quantization indices
• Slepian-Wolf limit applied to the same
• Oohamas outer bound

13
Jointly Gaussian Sources
Yt
Xt
14
Jointly Gaussian Sources
Yt
Xt
15
Oohamas Outer Bound

• For jointly Gaussian (X, Y) with unit variance
  and correlation coefficient r,

16
Comparison of Bounds
17
Comparison of Bounds
18
Comparison of Bounds
19
Outline

• A simple coding strategy based on bipartite graph
  coloring
• Comparison of achieved rates to the bounds
• Performance of non-uniform quantization
• Why compandors (compressor expandor) do not
  help
• Power of expansors (expandor compressor) for
  jointly Gaussian sources

20
A Simple Coloring Strategy

• For a fixed number of cells N, and covering belt
  width W,

• Add vertices and edges to the graph to ensure
  that WNY divides N?, of vertices, and every
  vertex sees W (circular) neighbors on the other
  side

• Color Y-vertices with cY(j) j mod WNY

• Translate X-coloring to one ball from each urn
  problem and minimize the X-color entropy.

21
Graph Completion
Choose NY 2
W 3
N 11
22
Creation of Urns
23
Creation of Urns
24
Creation of Urns
25
X-Balls to Urns
11
10
9
8
7
6
5
4
3
2
1
0
26
X-Balls to Urns
11
10
9
8
6
5
4
3
2
0
1
7
27
X-Balls to Urns
11
10
9
8
6
5
4
3
2
0
1
7
28
X-Balls to Urns
11
10
9
6
5
4
3
0
1
2
7
8
29
X-Balls to Urns
1
2
3
4
5
6
7
8
9
10
11
0
30
Super-urns
31
X-ColoringOne Ball from Each Urn
0
5
7
1
4
10
11
2
8
6
3
9
5
0
3
1
7
8
11
6
9
10
2
4
32
X-ColoringOne Ball from Each Urn
5
0
3
1
7
8
11
6
9
4
2
10
33
X-ColoringOne Ball from Each Urn
5
0
3
1
7
11
8
6
9
4
10
2
34
X-ColoringOne Ball from Each Urn
5
6
3
1
7
11
8
0
9
4
10
2
Which choice minimizes entropy?
35
Lemma

• For each sub-problem (super-urn), the entropy of
  the color classes is minimized by rank-based
  combination of balls.
• Repeat until no balls left
• Pick balls with largest probability from each
  urn, and assign them an X-color

36
Performance Comparisons
37
Performance Comparisons
38
Performance Comparisons
39
Outline

• A simple coding strategy based on bipartite graph
  coloring
• Comparison of achieved rates to the bounds
• Performance of non-uniform quantization
• Why compandors (compressor expandor) do not
  help
• Power of expansors (expandor compressor) for
  jointly Gaussian sources

40
Companding 101
g(x)
x
41
Companding 101
g(x)
x
For fixed N, D ?, H ?
It turns out that this does not reduce D ?H
42
Companding in Our Framework
Yt
Xt
43
High Resolution Companding
Yt
Covering belt

• For fixed D,
• N ?
• W ?
• NW ? (FL rates) ?
• Achieved VL rates ?

Xt
44
Expansors
Yt
Xt
45
High Resolution Expansing
Yt
Covering belt

• For fixed D,
• N ?
• W ?
• NW ? (FL rates) ?
• Achieved VL rates ?

Xt
46
Performance Comparisons
47
Performance Comparisons
48
Performance Comparisons
49
Comparison of Total Rate Loss
50
Comparison of Total Rate Loss
51
Comparison of Total Rate Loss
52
Summary Conclusions

• Scalar DSC is treated as a zero-error coding
  problem for the covering belt.
• A simple bipartite coloring algorithm comes very
  close to zero-error coding bounds for Gaussian
  sources under uniform quantization.
• Exploiting the geometry in non-uniform
  quantization in the form of expansing yields
  reduced rates.