Distributed Video Coding

1
Distributed Video Coding
VLBV, Sardinia, September 16, 2005 Bernd
Girod Information Systems Laboratory Stanford
University
2
Outline

• Foundations of distributed coding
• Slepian-Wolf Theorem and practical Slepian-Wolf
  coding
• Wyner-Ziv results and practical Wyner-Ziv coding
• Low-complexity video encoding
• Pixel-domain and transform-domain coding
• Hash-based receiver motion estimation
• Wyner-Ziv residual coding
• Error-resilient video transmission
• Systematic lossy joint source-channel coding
• Improving the error-resiliency of MPEG (or
  anything else) by Wyner-Ziv coding

3
Outline

• Foundations of distributed coding
• Slepian-Wolf Theorem and practical Slepian-Wolf
  coding
• Wyner-Ziv results and practical Wyner-Ziv coding
• Low-complexity video encoding
• Pixel-domain and transform-domain coding
• Hash-based receiver motion estimation
• Error-resilient video transmission
• Systematic lossy joint source-channel coding
• Improving the error-resiliency of MPEG by
  Wyner-Ziv coding

4
Compression of Dependent Sources
Source X
X
Joint Encoder
Joint Decoder
Y
Source Y
5
Distributed Compression of Dependent Sources
Source X
Encoder X
X
Joint Decoder
Y
Source Y
Encoder Y
6
Slepian Wolf Theorem
Achievable rate region for i.i.d sequences
Independent decoding
7
Slepian Wolf Theorem
Achievable rate region for i.i.d sequences
Independent decoding No errors
Joint decoding Vanishing error probabilityfor
long sequences
Slepian, Wolf, 1973
8
Lossless Compression with Receiver Side
Information
Source
Decoder
Encoder
9
Distributed Compression and Channel Coding
Source XY
Encoder
Decoder
P

• Idea
• Interpret Y as a noisy version of X with
  channel errors D
• Encoder generates parity bits P to protect
  against errors D
• Decoder concatenates Y and P and performs
  error-correcting decoding

10
Towards Practical Slepian-Wolf Coding

• Convolution coding for data compression Blizard,
  1969, unpublished
• Convolutional source coding Hellman, 1975
• Coset codes Pradhan and Ramchandran, 1999
• Trellis codes Wang and Orchard, 2001
• Turbo codes
• Garcia-Frias and Zhao, 2001
• Bajcsy and Mitran, 2001
• Aaron and Girod, 2002
• LDPC codes Liveris, Xiong, and Georghiades,
  2002
• . . .
• . . .

11
Slepian-Wolf Coding Using Turbo Codes
Parity bits
Parity bits
Aaron and Girod, 2002
12
Lossy Compression with Side Information
Source
Encoder
Decoder
Wyner, Ziv, 1976 For mse distortion and
Gaussian statistics, rate-distortion functions
of the two systems are the same.
Source
Encoder
Decoder
13
Practical Wyner-Ziv Encoder and Decoder
Wyner-Ziv Decoder
Wyner-Ziv Encoder
Slepian- Wolf Decoder
Minimum Distortion Reconstruction
Slepian-Wolf Encoder
Quantizer
14
Non-Connected Quantization Regions

• Example Non-connected intervals for scalar
  quantization
• Decoder Minimum mean-squared error
  reconstruction with side information

x
x
15
Outline

• Foundations of distributed coding
• Slepian-Wolf Theorem and practical Slepian-Wolf
  coding
• Wyner-Ziv results and practical Wyner-Ziv coding
• Low-complexity video encoding
• Pixel-domain and transform-domain coding
• Hash-based receiver motion estimation
• Error-resilient video transmission
• Systematic lossy joint source-channel coding
• Improving the error-resiliency of MPEG by
  Wyner-Ziv coding

16
Interframe Video Coding
PredictiveInterframe Encoder
PredictiveInterframe Decoder
X
X
Side Information
17
Video Coding with Low Complexity
Wyner-ZivIntraframe Encoder
Wyner-ZivInterframe Decoder
X
X
Side Information
Witsenhausen, Wyner, 1978 Puri, Ramchandran,
Allerton 2002 Aaron, Zhang, Girod, Asilomar
2002
18
(No Transcript)
19
Low Complexity Encoding and Decoding
20
Pixel Domain Wyner-Ziv Coder
Interframe Decoder
Intraframe Encoder
Slepian-Wolf Codec
Turbo Decoder
Turbo Encoder
Scalar Quantizer
Buffer
Request bits
previous
Interpolation
Key frames
next
Aaron, Zhang, Girod, Asilomar 2002 Aaron,
Rane, Zhang, Girod, DCC 2003
21
Pixel Domain Wyner-Ziv Coder
After Wyner-Ziv Decoding
Decoder side informationgenerated by
motion-compensated interpolationPSNR 30.3 dB
16-level quantization 1.375 bpp11 pixels in
errorPSNR 36.7 dB
22
Pixel Domain Wyner-Ziv Coder
After Wyner-Ziv Decoding
Decoder side informationgenerated by
motion-compensated interpolationPSNR 24.8 dB
16-level quantization 2.0 bpp0 pixels in
errorPSNR 36.5 dB
23
Stanford Camera Array
Courtesy Marc Levoy, Stanford Computer Graphics
Lab
24
Distributed Compression
Wyner-Ziv Cameras
Conventional Cameras


Distributed Encoding
WZ-ENC
WZ-ENC
WZ-DEC
WZ-DEC
Geometry Reconstruction
Joint Decoding
Rendering
Zhu, Aaron, Girod, 2003
25
Light Field Compression
Wyner-Ziv, Pixel-Domain
JPEG-2000
Rate 0.11 bpp PSNR 39.9 dB
Rate 0.11 bpp PSNR 37.4 dB
26
DCT-Domain Wyner-Ziv Video Encoder
Turbo Encoder
Extract bit-planes
level Quantizer
Wyner-Ziv parity bits
DCT
Input video frame

bit-plane Mk
For each low frequency coefficient band k
Previous-frame quantized high freq coefficients
Comparison
High frequency bits
Entropy Coder
Quantizer
27
Wyner-Ziv Video Decoder with Motion Compensation
Decoded frame
For each low frequency band
Reconstruction
Turbo Decoder
Wyner-Ziv parity bits
IDCT
Refinedside information
Side information
DCT
DCT
Motion-compensated Extrapolation
ExtrapolationRefinement
Previous frame
Entropy Decoder and Inverse
Quantizer
High frequency bits
Reconstructed high frequency coefficients
28
Rate-Distortion Performance - Salesman

• Every 8th frame is a key frame
• Salesman QCIF sequence at 10fps 100 frames

29
Rate-Distortion Performance Hall Monitor

• Every 8th frame is a key frame
• Hall Monitor QCIF sequence at 10fps 100 frames

30
Rate-Distortion Performance Foreman

• Every 8th frame is a key frame
• Foreman QCIF sequence at 10fps 100 frames

31
Salesman at 10 fps
DCT-based Intracoding 149 kbps PSNRY30.0 dB
Wyner-Ziv DCT codec 152 kbps PSNRY35.6 dB
GOP8
32
Hall Monitor at 10 fps
DCT-based Intracoding 156 kbps PSNRY30.2 dB
Wyner-Ziv DCT codec 155 kbps PSNRY37.1 dB
GOP8
33
Foreman at 10 fps
DCT-based Intracoding 290 kbps PSNRY34.4 dB
Wyner-Ziv DCT codec 293 kbps PSNRY35.5 dB
GOP8
34
Wyner-Ziv Residual Coding
Wyner-ZivDecoder
Wyner-Ziv Encoder
Xn
Side Information
35
Rate-Distortion Performance Foreman

• Every 8th frame is a key frame
• Foreman QCIF sequence at 30fps 100 frames

36
Rate-Distortion Performance Foreman

• Every 8th frame is a key frame
• Foreman QCIF sequence at 30fps 100 frames

37
Outline

• Foundations of distributed coding
• Slepian-Wolf Theorem and practical Slepian-Wolf
  coding
• Wyner-Ziv results and practical Wyner-Ziv coding
• Low-complexity video encoding
• Pixel-domain and transform-domain coding
• Hash-based receiver motion estimation
• Error-resilient video transmission
• Systematic lossy joint source-channel coding
• Improving the error-resiliency of MPEG by
  Wyner-Ziv coding

38
Systematic Lossy Error Protection (SLEP)of
Compressed Video
Analog channel (uncoded)
Any OldVideo Encoder
Video Decoder with Error Concealment
S
S
Error-Prone channel
Aaron, Rane, Girod, ICIP 2003

• Graceful degradation without a layered signal
  representation

39
MPEG with Systematic Lossy Error Protection
Rane, Aaron, Girod, VCIP 2004
40
Reed-Solomon Coding Across Slices
RS code across slices
41
Results CIF Foreman
FECSLEP
Main Stream _at_ 1.092 Mbps FEC (n,k) (40,36)
FEC bitrate 120 Kbps Total 1.2 Mbps WZ
Stream _at_ 270 Kbps SLEP (n,k) (52,36) WZ
bitrate 120 Kbps Total 1.2 Mbps

SLEP
FEC
42
MPEG with Systematic Lossy Error Protection
main
S
MPEG Encoder
Channel
Parityonly
R-S Encoder
Rane, Aaron, Girod, VCIP 2004
43
SLEP MPEG Codec with Simple Decoder
44
Performance at Symbol Error Rate 10-4
MPEG-2 video 2 Mbps FEC 222 Kbps (PSNR 29.78
dB)
MPEG-2 video 2 Mbps Wyner-Ziv 222 Kbps (PSNR
35.45 dB)
45
Distributed Coding of VideoWhy Should We Care?

• Chance to reinvent compression from scratch
• Entropy coding
• Quantization
• Signal transforms
• Adaptive coding
• Rate control
• . . .
• Enables new compression applications
• Very low complexity encoders
• Compression for networks of cameras
• Error-resilient transmission of signal waveforms
• Digitally enhanced analog transmission
• Unequal error protection without layered coding
• . . .

46
The End

• Further interest
• B. Girod, A. Aaron, S. Rane, D. Rebollo-Monedero,
  "Distributed Video Coding," Proceedings of the
  IEEE, Special Issue on Video Coding and Delivery.
  January 2005. http//www.stanford.edu/bgi
  rod/pdfs/DistributedVideoCoding-IEEEProc.pdf