Distributed Source Coding for Image and Video Applications

1
Distributed Source Coding for Image and Video
Applications

• Ngai-Man (Man) CHEUNG
• Signal and Image Processing Institute
• University of Southern California
• Los Angeles, CA, USA
• (This work was supported in part by NASA-JPL)

2
Acknowledgements

• USC
• Antonio Ortega
• Huisheng Wang
• Caimu Tang
• NASA-JPL
• Sam Dolinar
• Aaron Kiely
• Hua Xie
• Matthew Klimesh

3
Overview of This Talk
New image and video compression algorithms based
on distributed source coding (DSC)
Parallel Hyperspectral Image Encoding
Flexible Video Decoding
Spectral Band
?f
We will demonstrate DSC based algorithms can
overcome some shortcomings in closed-loop
prediction (CLP)
4
Motivation
5
Compression with Closed-loop Prediction (CLP)
E.g., MPEG, H.26X
Prediction residue
ZY-X

X
X
Encoder
Decoder
Y
Exactly same predictor

• Predictor Y
• Motion-compensated predictor in neighboring
  frames
• Co-located pixels in neighboring slices in
  volumetric image

6
Issues in Closed-loop Prediction High
Complexity Encoding

• High complexity encoding
• Motion estimation
• Some emerging applications require low complexity
  encoding, e.g., mobile video, video sensors

time
Y
X
Reference frame
Current frame
7
Issues in Closed-loop PredictionVulnerable to
Transmission Error

• Error in predictor propagates to subsequent
  frames drifting

time
error
8
Issues in Closed-loop PredictionLack of
Decoding Flexibility

• Some emerging applications need to support
  multiple decoding paths, e.g., multiview video
• When users can choose among different decoding
  paths, it is not clear which previous
  reconstructed frame will be available to use in
  the decoding

View
View
View
Y1
Y2
Y2
Y0
Y0
Y1
Y0
Y2
Y1
Time
X
X
X
Decoding path
9
Issues in Closed-loop PredictionInherent
Sequential

• Difficult to achieve parallel encoding of slices
  in volumetric image compression

Processor
Processor
Spectral Band
Processor
Processor
New compression algorithms based on distributed
source coding
10
Distributed Source Coding
11
Distributed Source Coding
Lossless compression of random variable X - Intra
coding R H(X)
(i) CLP (e.g. DPCM)
ZX-Y
X
X
RH(XY)
Encoder
Decoder
Y
(ii) DSC

X
X
X
Encoder
Decoder
Y
P(X,Y)

• Encoding does not require Y

12
Distributed Source Coding
Lossless compression of random variable X - Intra
coding R H(X)
(i) CLP (e.g. DPCM)
ZX-Y
X
X
RH(XY)
Encoder
Decoder
Y
(ii) DSC

X
X
X
Encoder
Decoder
Y
P(X,Y)

• Encoding does not require Y

13
Distributed Source Coding
Lossless compression of random variable X - Intra
coding R H(X)
(i) CLP (e.g. DPCM)
ZX-Y
X
X
RH(XY)
Encoder
Decoder
Y
(ii) DSC

X
X
X
Encoder
Decoder
Y
P(X,Y)

• Encoding does not require Y

14
Why DSC?When predictor is not accessible during
encoding

• Low complexity video encoding
• Due to complexity constraint at the encoder

• Sensors network, wireless camera
• Correlated inputs are encoded at different
  spatial locations

time
X
Y
X
Y
DSC

X
X
X
Encoder
Decoder
Y
P(X,Y)
How to estimate this correlation?
15
Why DSC?To decouple compressed data from
predictor
In CLP, prediction residue is closely coupled
with the predictor
(i) CLP (e.g. DPCM)
ZX-Y
X
X
Encoder
Decoder
Y
Y
(ii) DSC

X
X
X
Encoder
Decoder
How to encode X?
Y
P(X,Y)
In DSC, compressed data is computed based on the
correlation
More robust to uncertainty on or error in Y
16
DSC Example Using Coset

• X takes value 0, 255 (uniformly)
• Intra coding requires 8 bits for lossless
• Correlation P(X,Y) 4 gt Y-X -4 (i.e., Y-X
  takes value in -4,4))
• Can explore this correlation
• If Y is available at both the encoder and decoder
• CLP communicate the residue

Requires 3 bits to convey X
If Y is not available at the encoder, how can we
communicate X?
17
DSC Example Using Coset (Contd)

• How to communicate X if Y is not available at the
  encoder?
• Partition reconstruction levels into different
  cosets
• Each coset includes several reconstruction levels
• Encoder transmit coset label (syndrome), need 3
  bits
• Decoder disambiguate label (closest to Y)

X2
P(X,Y) 4 gt Y-X -4
Encoder
A B C D E F G H A B C
D G H
C
C
Decoder
Y
C
C
C

Requires 3 bits to convey X
18
Correlation information is important in DSC

• Correlation information determines the number of
  coset for error-free reconstruction

Encoder Over-estimate the correlation and use
fewer cosets
X2
A B C D A B C D A B C
D .
C C
C
Decoder
Y
C C
C
C
Decoding error may occur if encoder
over-estimates the correlation
19
DSC - Main Ideas

• Encoding Send ambiguous information to achieve
  compression
• E.g., one label represents group of
  reconstruction levels
• Decoding information is disambiguated at the
  decoder using side information Y
• Pick the coset member closest to Y
• Level of ambiguity (hence the encoding rate) is
  determined by correlation between X and Y
• More correlated, more ambiguous representation
  using less bits

How about the coding efficiency?
20
DSC Theoretical Performance
Lossless compression of random variable X

• CLP

RH(XY)
X
X
Encoder
Decoder
Y

• DSC

Slepian, Wolf 1973
Efficient encoding possible even when encoder
does not have precise knowledge of Y
21
DSC for Practical Problems

• In practical applications, can DSC achieve the
  same coding performance as CLP?
• How to design a whole system based on DSC?
• What are the advantages to use DSC?

22
Application I DSC-based Hyperspectral Image
Compression
23
Hyperspectral Image

• Consist of hundreds of spectral bands (slices)
• Data-size non-trivial, e.g., AVIRIS (NASA) more
  than 140M bytes/image
• Captured by systems with limited resources
  (satellites or spacecrafts)
• Encoding complexity important
• High speed encoding desirable
• Parallel encoding

Relative brightness
Spectral Band
Spectrum
?f
wavelength
24
Related Work

• Spectral bands are correlated
• Inter-band prediction approaches, i.e., CLP,
  e.g., Miguel, Askew, Chang, Hauck, Ladner,
  Riskin DCC 04
• Use previous reconstructed band to predict
  current band
• Encode the prediction residue
• Inherently sequential

Spectral Band
?f
25
CLP Inherently Sequential

• CLP Same predictor has to be used in encoding
  and decoding

Miguel, Askew, Chang, Hauck, Ladner, Riskin DCC
04
Encoder
Decoding at encoder
Same predictor
Decoder
26
Related Work

• Spectral bands are correlated
• Inter-band prediction approaches, i.e., CLP,
  e.g., Miguel, Askew, Chang, Hauck, Ladner,
  Riskin DCC 04
• Use previous reconstructed band to predict
  current band
• Encode the prediction residue
• Inherently sequential
• 3D wavelets approaches, e.g., Tang, Cho,
  Pearlman ICIP 03
• 3D-SPECK, 3D-SPIHT, 3D-ICER (NASA JPL)
• Spatial and cross-bands filtering
• Most energy concentrate in low pass sub-bands
  (spatially and spectrally)
• Expensive memory requirement
• Our goal parallel encoding of each band
• minimum inter-processor communication
• modest memory requirement

27
DSC Based Hyperspectral Image Compression
System Overview

• Encoder consists of multiple processors, each
  compresses one spectral band

Processor
Parallel encoding
Y
Processor
How to estimate this correlation?
Spectral Band
P(X, Y)

X
X
Processor
Model-based estimation
Processor

• Estimate P(X, Y) could be non-trivial
• Limited communication bandwidth between
  processors
• Data exchange could make parallel encoding
  inefficient

28
DSC Based Hyperspectral Image Compression
Encoder Block Diagram
Bit-plane Extraction
Input Buffer
Linear
Wavelet
Predictor
Transform
Y
Yb
Processor k-1
How to estimate these crossover probabilities
efficiently?
Processor k
PrXb ? Yb
Sign/refinement bit-planes
Slepian-Wolf Coder
X
Xb
Wavelet
Bit-planes Extraction
Input Buffer
multiplexing
Transform
Zerotree Coder
Significance bit-planes
Adaptive coding

• Side-information corresponding bit-plane in
  previous band
• Correlation crossover probability

Tang, Cheung, Ortega, Raghavendra DCC 05,
Cheung, Tang, Ortega, Raghavendra EURASIP J.
Signal Processing 06
29
Correlation Estimation
30
Direct Approach to Estimate Correlation
Explicitly generate SI
Bit-plane Extraction
Input Buffer
Linear
Wavelet
Predictor
Transform
Y
Yb
Processor k-1
Significance map
Processor k
Direct Comparison
PrXb ? Yb
Sign/refinement bit-planes
Slepian-Wolf Coder
X
Xb
Wavelet
Bit-planes Extraction
Input Buffer
multiplexing
Transform
Zerotree Coder
Significance bit-planes
Idle
Processor k-1
Processor k
Idle

• Inefficient parallel encoding

31
Model-based Approach to Estimate Correlation
Do not generate SI
Input Buffer
Processor k-1
Sampling with small amount of pixels
Cheung, Ortega ICIP 06
Cheung, Wang, Ortega IEEE Trans. IP 08
Estimate correlation noise using pixels
Processor k
Linear
Estimate Correlation Noise Model
Derive crossover analytically
Predictor
PrXb ? Yb
Estimate Source Model
Sign/refinement bit-planes
Slepian-Wolf Coder
X
Xb
Wavelet
Bit-planes Extraction
Input Buffer
multiplexing
Transform
Zerotree Coder
Significance bit-planes
Processor k-1
Processor k

• Small amount of computation
• Small amount of communication overhead
• Minimum interaction between processors

32
Timing Diagrams Parallel Encoding of
Hyperspectral Image
(i) CLP
Transmit whole reconstructed band
time
(ii) DSC
Transmit partial original band
time
In DSC, encoding requires only correlation
information instead of the exact reconstructed
data
33
Model-based Estimation

• Two steps
• Step 1 - Estimate model Estimate the model
  parameters (e.g., maximum likelihood estimation
  (MLE), )
• Step 2 - Derive bit-plane level statistics Use
  the estimated model to derive the crossover
  probabilities analytically

34
Step 1 - Estimate Model

• Estimate joint p.d.f. fXY(x,y)
• X Transform coefficients of current spectral
  band
• Y Transform coefficients of previous spectral
  band
• Assume YXZ
• Z is the correlation noise independent of X
• Factorize fXY(x,y) fX(x) fZ(y-x)
• fX(x) source model
• fZ(z) correlation noise model
• Estimate fX(x), fZ(z) with different procedures

Y
Z
X
35
Estimate Models in Wavelet-based DSC

• Source model fX(x)
• Low levels different Laplacian models for
  different sub-bands
• Estimated by M.L.E.
• High levels empirical p.m.f.
• Partition determined by no. of coefficients in a
  sub-band
• M.L.E. estimator distributed normally
• Correlation noise model fZ(z)
• Single Laplacian for all coefficients
• Estimate variance using pixel domain data
• 9/7 filter variance of pixel data almost the
  same as that of transform coefficients Usevitch
  DCC 96
• 3.125 samples

High (decomposition) levels sub-bands Empirical
p.m.f. for source
Y
Z
X
Low levels sub-bands Laplacian source modeling
36
Step 2 - Derive Bit-plane Crossover Probability

• Derive estimate of pl (Given fXY(x,y))
• Crossover of bit-plane data correspond to regions
  Ai
• Tracing when bit crossover happens
• Crossover probability estimate
• In practice, only have to sum over a few regions

Sample space of X and Y
Bits crossover
Y
4 5 6 7


0 0 0 0
1 1 1 1
0 0 1 1
0 1 0 1
0 1 2 3


1 0 0 0 0
0 0 0 0 0
0 0 0 1 1
0 0 1 0 1
8 9


1 1 1 1
0 0 0 0
0 0 1 1
0 1 0 1
X
Xb?Yb
msb
37
Derive Bit-plane Crossover Probability
Sign/Refinement

• Derive crossover probabilities of refinement/sign
  bit-plane

Crossover in refinement bit-plane
Crossover in sign bit-plane
Yi

A1

3x2l
A3
2x2l
A0
2l
A2
Xi
0
2l
2x2l
3x2l
Xi is already significant
Xi is becoming significant
38
Derive Bit-plane Crossover Probability
Sign/Refinement

• Derive crossover probabilities of refinement/sign
  bit-plane

Crossover in refinement bit-plane
Crossover in sign bit-plane
Yi

A1

3x2l
A3
2x2l
A0
2l
A2
Xi
0
2l
2x2l
3x2l

• Overall crossover for entire image
• Weight each integral when combining them

fXi(x)
Expected proportion of refinement bits
Expected proportion of sign bits
?2

?0
?1
2l
Xi
2l1
39
Derive Bit-plane Crossover Probability High
Decomposition Level Sub-bands

• Characterize the sources by empirical p.m.f.

Crossover in refinement bit-plane
fZ(z)
Crossover in sign bit-plane
40
Coding Strategy and Adaptive Coding Scheme
41
Coding Strategy

• DSC (inter-band coding) or zerotree (intra)
  coding?

Processor k
PrXb ? Yb
Sign/refinement bit-planes
Slepian-Wolf Coder
X
Xb
Wavelet
Bit-planes Extraction
Input Buffer
multiplexing
Transform
Zerotree Coder
Significance bit-planes
Adaptive coding

• Efficiency of ZTC Marginal distribution of
  source
• Efficiency of DSC Joint distribution between
  source and SI

42
Encode Significance Map by ZTC
Marginal
fXi(x)
s s s s s s

1
0 1
0 1 1
1 1 1 1 1
0 0 1 0 1 0
ZTC becomes less efficient
At significance level l
2l1
2l
Xi
p(b0)
p(b1)
Significance map

Each column is a wavelet coefficient
Decrease l
fXi(x)
Refinement bit

• Significance map is biased toward zero
• ZTC can achieve compression
• When l is small, zero-bias decrease
• Zero-bias decreases as variance increases

2l1
2l
Xi
p(b0)
p(b1)
43
Encode Significance Map by DSC
Yi
Joint p.d.f.
Bit crossover

s s s s s s

1
0 1
0 1 1
1 1 1 1 1
0 0 1 0 1 0
DSC becomes less efficient

3x2l
2x2l
At significance level l
2l
Xi
0
2l
2x2l
3x2l
Decrease l
Significance map

Bit crossover
Yi

Refinement bit

• Significance map can be compressed by DSC
• When l is small, size of each crossover region
  decreases
• more samples occurred inside

3x2l
2x2l
2l
Xi
0
2l
2x2l

• Non-trivial to choose between ZTC or DSC
• Modeling technique to determine the strategy to
  switch between ZTC and DSC

44
Modeling Techniques to Determine Switching
Between ZTC and DSC

• Estimate zero-bias from marginal distribution
• No. of coded bit when using ZTC

• Estimate crossover probability from joint p.d.f.
• No. of coded bit when using DSC

NZTC(q)
NDSC(pl)
fXi(x)
2l1
2l
Xi
qPrs0
Prs1

• At each significance level
• For each wavelet subband

45
Number of Coded Bits Estimated by Modeling
Number of coded bits (estimated)
Number of coded bits (estimated)
High level subband
Low level subband
ZTC
ZTC
DSC
DSC
Significance level
Significance level
Switching threshold
Switching threshold
There are thresholds at which encoding should
switch from ZTC to DSC
46
Adaptive Coding Scheme
Number of coded bits (estimated)
Number of coded bits (estimated)
High level subband
Low level subband
ZTC
ZTC
DSC
DSC
Significance level
Significance level
Switching threshold
Switching threshold
s s s s s s

1
0 1
0 1 1
1 1 1 1 1
0 0 1 0 1 0

• Different wavelet sub-bands have different
  switching thresholds
• Optimal switching thresholds estimated by modeling

Most significant bit
Initially use ZTC
Switch to use DSC
Least significant bit
Zerotree coded
DSC coded


47
Adaptive Coding Scheme
Non-adaptive significance info. is always
zerotree coded
Adaptive Initially ZTC for signif. Info., later
switch to use DSC
sign
s s s s s s

1
0 1
0 1 1
1 1 1 1 1
0 0 1 0 1 0
s s s s s s

1
0 1
0 1 1
1 1 1 1 1
0 0 1 0 1 0
Most significant bit
Least significant bit
Each column is a wavelet coefficient
Switch to use DSC
Zerotree coded

• Different wavelet sub-bands have different
  switching thresholds
• Optimal switching thresholds estimated by modeling

Slepian-Wolf coded

48
Results Coding Performance

• Comparison with 3D Wavelet based algorithms
• 3D ICER from NASA JPL Kiely, Klimesh, Xie,
  Aranki 2006
• Our proposed algorithm is comparable to one
  version of 3D Wavelet

3D Wavelet (JPL)
DSC (Adaptive)
SPIHT
49
Results Coding Performance (Contd)
DSC (Adaptive)
DSC (Non-adaptive)
SPIHT
50
Results Coding Performance (Contd)

• Comparison with CLP
• CLP Use SPIHT to encode prediction residue

DSC
CLP
51
Summary DSC Hyperspectral Image Compression

• DSC based hyperspectral image compression
• Parallel encoding with modest inter-processors/mem
  ory overhead
• Model based estimation
• Adaptive coding scheme
• Coding performance comparable to that of
  non-distributed schemes

52
Application IIDSC Approach to Flexible Video
Decoding
53
Flexible Video Decoding
Apply DSC to generate a single bitstream that can
be decoded in several different orders - Decoder
can choose among multiple decoding paths

• Free viewpoint switching in multiview video
  compression
• Forward and backward video playback
• Robust video transmission

54
Flexible Decoding Example Free Viewpoint
Switching in Multiview Video
Cheung, Ortega PCS 07
User selects a particular view
User switches between different views during
playback
View
Y1
Y0
Y2
Y2
Y2
Y0
Y0
Y1
Y1
Time
X
X
X
video frame
Decoding path
In order to address viewpoint switching,
compression schemes have to support multiple
decoding paths
55
Flexible decoding poses challenges tovideo
compression
When users can choose among different decoding
paths, it is not clear which previous
reconstructed frame will be available to use in
the decoding
View
Y1
Y0
Y2
Y2
Y2
Y0
Y0
Y1
Y1
Time
X
X
X
Either Y0 or Y1 or Y2 will be available at decoder
Uncertainty on predictor status at decoder!!!!
Multiple decoding paths
56
Problem Formulation Flexible Decoding
To support flexible decoding, encoder needs to
operate under uncertainty on decoder predictor
Cheung, Ortega MMSP 07
Encoder
Decoder
Encoder does not know which Yi will be available
at decoder
Y1
Y2
Y0
Y2
Y0
Y1

X
X

• Assume feedback is not available
• E.g., Pre-encoded video data for storage media

57
Flexible Decoding Example Forward/Backward
Frame-by-frame Video Playback
Cheung, Wang, Ortega VCIP 06
User can choose to play back in either direction
Either past or future reconstructed frame will
be available at decoder
Y0
time
X

Y1
Decoding path
Y0, Y1 best motion-compensated predictor for X

• Not B-frame (as in video coding standards)
• Not multiple reference frames

58
Flexible Decoding Example Robust Video
Transmission
Wang, Prabhakaran, Ramchandran ICIP 06
Y1
Y0
time
X

error free
error corrupted
Decoding path
Some reference frames have error, but encoder
does not know which one (assume feedback is not
available)
59
Address Flexible Decoding Within Closed-Loop
Predictive (CLP) Coding Framework
With CLP approach (e.g., MPEG/H.26X), encoder has
to send multiple prediction residues to decoder
Prediction residue is tied to a specific
predictor
Encoder
Decoder
Y1
Y2
Y0
Y2
Y0
Y1



Z0, Z1, Z2
Z1X-Y1


XY2Z2
Z2X-Y2
Z0X-Y0

X
X
Zi P-frame or SP-frame

Overhead increases with the number of predictor
candidates
X may not be identical when different Yi are used
as predictor - drifting
60
Our Contributions

• Propose distributed source coding (DSC) based
  coding algorithm to address flexible decoding
• Apply DSC to generate a single bitstream that can
  be decoded in several different orders
• Propose an efficient encoding algorithm
• Macroblock mode
• Significance coding
• Correlation model
• Minimum MSE dequantization

61
Related Work

• Differences between DSC-based low complexity
  encoding Puri, Ramchandran Allerton 02,
  Aaron, Zhang, Girod Asilomar 02

DSC-based low complexity encoding DSC-based flexible decoding
Key objective Low complexity encoding for mobile video Generate a single bitstream to support multiple decoding paths
Encoder complexity Require low-complexity, real-time encoding Not primary issue. Applications use off-line encoding
Encoder access to SI SI not accessible Encoder has access to all SI
62
Related Work

• Provide random access in compression of
  image-based rendering data/multiview video
• Jagmohan, Sehgal, Ahuja Asilomar 03
• Aaron, Ramanathan, Girod MMSP 04
• Guo, Lu, Wu, Gao, Li VCIP 06
• All assume encoder has knowledge on predictor
  status, e.g., using feedback
• Robust video transmission Wang, Prabhakaran,
  Ramchandran ICIP 06
• Specific algorithms used are different
• Improve decoding flexibility and accessibility
  Naman, Taubman ICIP 07
• Feedback and conditional replenishment
• Information theoretic results Draper, Martinian
  ISIT 07
• No practical coding algorithm proposed
• Forward/backward video playback Cheung, Wang,
  Ortega VCIP 06
• Considerably different algorithm to achieve
  better coding performance

63
Solution Based on Distributed Source Coding
64
DSC - Virtual Communication Channel Perspective
In DSC, encoder can communicate X by sending
parity information (E.g., Girod, Aaron, Rane,
Rebollo-Mondero Proc. IEEE 04)
CLP
DSC
Z
ZX-Y

Y

X
X
X
X

Dec
Enc
Dec
Parity information
Y
Parity information is independent of a specific
predictor - What matters is the amount of parity
information
65
Address Flexible Decoding Using DSC
Under predictor uncertainty, encoder can
communicate X by sending an amount of parity
corresponding to the worst case correlation
Encoder
Decoder
Y1
Y2
Y0
Y1
Y0
Y2
fZ0(z)
fZ1(z)
fZ2(z)

Worst case parity
X
X
N virtual channels
Z0
Z1
Z2
Y0
Y1
Y2



X
X
X
X
X
X



Dec
Dec
Dec
Parity corresponding to the worst case
correlation noise
66
Flexible Decoding CLP vs. DSC
RCLP SH(Zi)
CLP
H(Z1)
H(Z2)
H(Z0)
Encoder
Decoder



CLP
Z0, Z1, Z2
Y1
Y2
Y0
Y1
Y0
Y2

X
X
DSC
Worst case parity
RDSC max H(Zi)
Bits required to communicate X with Yi at decoder
H(X Yi) H(Zi)
H(Z0)
DSC
H(Z1)
H(Z2)
67
Flexible Decoding CLP vs. DSC
RCLP SH(Zi)
CLP
H(Z1)
H(Z2)
H(Z0)
Encoder
Decoder



CLP
Z0, Z1, Z2
Y1
Y2
Y0
Y1
Y0
Y2

X
X
DSC
Worst case parity
RDSC max H(Zi)
Bits required to communicate X with Yi at decoder
H(X Yi) H(Zi)
H(Z0)
DSC
H(Z1)
H(Z2)
68
Flexible Decoding CLP vs. DSC
RCLP SH(Zi)
CLP
H(Z1)
H(Z2)
H(Z0)
Encoder
Decoder



CLP
Z0, Z1, Z2
Y1
Y2
Y0
Y1
Y0
Y2

X
X
DSC
Worst case parity
RDSC max H(Zi)
Bits required to communicate X with Yi at decoder
H(X Yi) H(Zi)
H(Z0)
DSC
H(Z1)
H(Z2)
69
Theoretical Performance
Cheung, Ortega VCIP 08
X, Yi i.i.d. discrete scalar sources
Encoder
Decoder
Y1
Y2
Y0
RX max H(XYi)
Y1
Y0
Y2

X
X
View
v-2 v-1 v v1 v2
CLP
Intra
3 1 0 2 4
Time
C
DSC
C current frame
70
Achievable Coding Rate
Same code for N source networks possible
Csiszar, Korner 1980
RX H(XY1)
RX H(XYN-1)
RX H(XY0)
X
X
X
Y0
Y1
YN-1
source

encoder
destination



X Y1
X YN-1
X Y0
71
Encoding Algorithm
72
Encoding Algorithm Motion Estimation and
Macroblock Classification
M may be classified to be in a skip mode if the
differences between M and predictors from some fi
are small - Skip At least one of the residue
blocks is a zero block after quantization
Input current frame
Motion vector information
M
Motion estimation/ MB classification
skip
CLP
z0, z1, z2, , zN-1
DSC
non-skip
f0, f1, f2, , fN-1
Candidate reference frames
Output bitstream
Mode decision
Majority using DSC
73
Encoding Algorithm DSC Coded MB
source
Direct coding
b(l)
Macroblock M
X
W
Quantization
Slepian-Wolf coding
DCT
Parity information
b(l)
Significance coding
s
Predictor from fi
ai
DCT
Model estimation
Yi
Side information candidate
K lowest frequency
W of k-th frequency
b(l)
Slepian-Wolf coding
Bit-plane extraction
74
Encoding Algorithm Significance Coding
Direct coding
b(l)
X
M
W
Quantization
Slepian-Wolf coding
DCT
Parity information
b(l)
Significance coding
s
Predictor from fi
ai
DCT
Model estimation
Yi
High frequency coefficients (kgtK)

• Expected number of source bits
  1pk(1Lk)(1-pk)
• Lead to source bits saving when pkgt1/Lk

W 0?
yes
W
b(l)
Bit-plane extraction
Slepian-Wolf coding
no
s
75
Drift-free Quantized coefficients in DSC coded
MB are exactly recovered
Direct coding
b(l)
X
M
W
Quantization
Slepian-Wolf coding
DCT
Parity information
b(l)
Significance coding
s
Predictor from fi
ai
DCT
Model estimation
Yi
Exactly recovered in decoding
Amount of output parity determined by the worst
case correlation
76
Models and Conditional Probability Estimation
Estimate p(b(l) Yi, b(l1), b(l2), ) by
integrating f(X Yi) as follow
p(W Yi)
f(X Yi)
b(l)0
b(l)1
p(b(l)0)
p(b(l)1)
W
X
Wr
Ws
Wt
Wu
Xr
Xs
Xt
Xu
Yi

W 2l1
Inverse quantization
77
Experiment Results - Multiview Video Coding
Allow switching from adjacent views three
predictor candidates
CLP
View
DSC
Y1
Y2
Y0
Time
Intra
X
Our proposed algorithm out-performs CLP and intra
coding
78
Experiment Results
CLP
DSC
Intra
79
Drifting Experiments
Our proposed algorithm is almost drift-free,
since quantized coefficients in DSC coded MB are
identically reconstructed
CLP
DSC
AkkoKayo (GOP30)
Drifting in CLP
Switching occurs
Switching occurs
View
View switching occurs at frame number 2
Time
80
Scaling Experiments

• Number of coded bits vs. number of predictor
  candidates

Bits per frame
v-1 v v1
Intra
CLP
1 0 2
DSC
3 4
5 7 6
Number of predictor candidates

• Bit-rate of DSC-based approach increases at a
  slower rate compared with CLP
• An additional candidate incurs more bits only if
  it has the worst correlation among all candidates

81
Experiment Results Forward/backward video
playback
Forward/backward playback two predictor
candidates
Intra
DSC
CLP
Inter-frame coding with one prediction residue
cannot support flexible decoding
Coastguard CIF
82
Application Scenarios Our proposed tools can
reduce decoding complexity in storage applications

• E.g., Pre-encoded video in storage media (say,
  DVD)
• Our proposed tools can reduce decoding complexity

Coding structure enabled by our proposed tools
Simulcast
View
View
v-1
v-1
v
v
I
I
I
I
P
P
P
S
Time
Time
P
P
P
P
Decoding path
P
P
S
P
Frames needed to be decoded
P
P
P
P
Switch-able frames coded by DSC
S
83
Summary DSC Application to Flexible Decoding

• DSC-based coding algorithm to address flexible
  decoding
• Single bitstream to support multiple decoding
  paths
• Parity information independent of a specific
  predictor
• Overhead depends on the worst correlation rather
  than the number of decoding paths
• Outperform CLP and intra coding in terms of
  coding performance
• Our proposed system is almost drift-free

84
Conclusions

• DSC based hyperspectral image compression
• DSC approach to flexible decoding
• In some applications, DSC based solution can
  address issues in conventional CLP coding
• Parallel encoding of volumetric image
• Flexible decoding of video data
• Decision between DSC and non DSC coding is very
  important
• Correlation estimation is very important

85
References

• Book Chapter/Journal Papers
• N.-M. Cheung and A. Ortega, Distributed
  Compression of Satellite Images, in Distributed
  Source Coding, Pier Luigi Dragotti and Michael
  Gastpar, Eds., Academic Press 2008.
• N.-M. Cheung and A. Ortega, Flexible Video
  Decoding A Distributed Source Coding Approach,
  in preparation for IEEE Transactions on Image
  Processing.
• N.-M. Cheung, H. Wang, A. Ortega, Sampling-based
  Correlation Estimation for Distributed Source
  Coding Under Rate and Complexity Constraints,
  IEEE Transactions on Image Processing 2008.
• N.-M. Cheung, C. Tang, A. Ortega and C. S.
  Raghavendra, Efficient Wavelet-based Predictive
  Slepian-Wolf Coding for Hyperspectral Imagery,
  EURASIP Journal on Signal Processing - Special
  Issue on Distributed Source Coding, 2006.
• H. Wang, N.-M. Cheung, A. Ortega, A Framework
  for Adaptive Fine Granularity Scalable Coding
  using Wyner-Ziv Techniques, EURASIP Journal on
  Applied Signal Processing, 2006.
• Conference Papers
• N.-M. Cheung and A. Ortega, Compression
  Algorithms for Flexible Video Decoding, in Proc.
  Visual Communications and Image Processing (VCIP)
  2008.
• N.-M. Cheung and A. Ortega, Distributed Source
  Coding Application to Low-Delay Free Viewpoint
  Switching in Multiview Video Compression,
  Picture Coding Symposium (PCS) 2007.
• N.-M. Cheung and A. Ortega, Flexible Video
  Decoding A Distributed Source Coding Approach,
  in Proc. IEEE International Workshop on
  Multimedia Signal Processing (MMSP) 2007.
• N.-M. Cheung and A. Ortega, A model-based
  approach to correlation estimation in
  wavelet-based distributed source coding with
  application to hyperspectral imagery, in Proc.
  IEEE Int. Conf. Image Processing (ICIP) 2006.
• N.-M. Cheung and A. Ortega, An efficient and
  highly parallel hyperspectral imagery compression
  scheme based on distributed source coding, in
  Asilomar Conf. Signals, Systems, and Computers
  2006.
• N.-M. Cheung, H. Wang, A. Ortega, Video
  compression with flexible playback order based on
  distributed source coding, in Proc. Visual
  Communications and Image Processing (VCIP) 2006.
  
• N.-M. Cheung, H. Wang, A. Ortega, Correlation
  estimation for distributed source coding under
  information exchange constraints, in Proc. IEEE
  Int. Conf. Image Processing (ICIP) 2005.
• C. Tang, N.-M. Cheung, A. Ortega, C. S.
  Raghavendra, Efficient inter-band prediction and
  wavelet based compression for hyperspectral
  imagery A distributed source coding approach,
  in Proc. IEEE Data Compression Conference (DCC)
  2005.
• H. Wang, N.-M. Cheung, A. Ortega, WZS Wyner-Ziv
  scalable predictive video coding, Picture Coding
  Symposium (PCS) 2004.

86
Conclusions

• DSC based hyperspectral image compression
• DSC approach to flexible decoding
• In some applications, DSC based solution can
  address issues in conventional CLP coding
• Parallel encoding of volumetric image
• Flexible decoding of video data
• Decision between DSC and non DSC coding is very
  important
• Correlation estimation is very important

Comments or Questions?
87
Parity information is independent of a specific
predictor
b(l)
1 0 0 1 1
1 1
SW Encoder




0 1 1




SW Decoder
b(l)
? ? ? ? ?
? ?
p(b(l) Yi, b(l1), b(l2), )