Distributed Video Coding with Unsupervised Learning of Motion Estimation

1
Distributed Video Coding with Unsupervised
Learning of Motion Estimation
EE 398B Project

• Young Min Kim
• Stephanie Kwan
• Karen Zhu

2
Outline

• Distributed Source Coding
• Wyner-Ziv Video Coder
• Distributed Stereo Image Coder
• Lossless Pixel Domain Distributed Video Coding
• Lossy Pixel Domain Distributed Video Coding
• Simulation Results
• Conclusion

3
Distributed Video Coding

• Conventional Video Coding
• High complexity encoder
• Low complexity decoder
• Distributed Video Coding
• Low complexity encoder
• High complexity decoder

4
Slepian-Wolf Theorem on Lossless Distributed
Coding
Separate Encoding
Joint Decoding
Decoder
(X,Y)
Encoder 1
X
(X,Y)
Y
Encoder 2
Slepian-Wolf Theorem
5
Wyner-Ziv Lossy Coding
Wyner-Ziv Coding
Separate Encoder
Joint Decoder
X
X
Y
Side Information available at Encoder
Joint Encoder
Joint Decoder
X
X
Y
Y
Wyner-Ziv Coding Performance
0
6
Wyner-Ziv Video Coder
Interframe Decoded
Intraframe Encoded
Decoded WZ Frames
WZ Frames
Slepian-Wolf Coder
Quantization
Reconstruction
q
Turbo Encoder
Turbo Decoder
S
S
Side Information
S
Request bits
Interpolation or Extrapolation
Conventional Intraframe Encoder
Conventional Intraframe Decoder
K
K
Decoded Key Frames
Key Frames
7
Distributed Compression of Stereo Images with
Unsupervised Learning
Request bits
LDPC Encoder
LDPC Decoder (M-step)
X
Termination Threshold
?
S
X
?
Disparity Estimator (E-step)
Y
8
Lossless Distributed Video Coder with
Unsupervised Learning of Motion
Request bits
LDPC Encoder
LDPC Decoder (M-step)
Termination Threshold
?
X
X
Decoded Frames
?
Motion Estimator (E-step)
Side Information
Y
Previous Reconstructed Frame
9
LDPC Coding
10
Lossless Distributed Video Coder with
Unsupervised Learning of Motion
Request bits
LDPC Encoder
LDPC Decoder (M-step)
Termination Threshold
?
X
X
Decoded Frames
?
Motion Estimator (E-step)
Side Information
Y
Previous Reconstructed Frame
11
2D Motion Estimation
12
Motion Vector Prediction (MVP)

• Change initial probability to the motion vector
  found from previous two frames

B
13
Lossy Distributed Video Coder
Intraframe Encoded
Interframe Decoded
Request bits
Reconstructed Frames
LDPC Encoder
LDPC Decoder (M-step)
Termination Threshold
Q
Q-1
S
S
Non-Key Frames
?
?
Side Information
Motion Estimator (E-step)
Previous Reconstructed Frame
Q
Conventional Intraframe Encoder
Conventional Intraframe Decoder
K
K
Decoded Key Frames
Key Frames
14
Comparison Schemes for Lossless Coding

• Proposed Schemes
• 2D motion estimation (2DME)
• 2D motion estimation motion vector prediction
  (MVP)
• Reference Schemes
• H(XY) Slepian-Wolf bound
• Motion estimation with motion oracle
• No motion estimation
• Intra-coding

15
Comparison Schemes for Lossy Coding

• Proposed Schemes (2DME MVP)
• 7 bits coder
• 6 bits coder
• 5 bits coder
• Reference Schemes
• H(XY) for given quantization level
• Motion estimation with motion oracle
• No motion estimation
• Intra-coding

16
Simulation Setting

• Foreman 65-95, Carphone 180-210
• 8 bitplanes for lossless, 5, 6, 7 bitplanes for
  lossy
• Frame size 72x88, Block size 8x8
• Motion vector between -5 and 5
• Initial probability
• 2DME - 0.75 at (0,0)
• MVP - 0.75 at previous motion vector

17
Average Rate for Lossless Distributed Video Coder
18
Sequence Rate Trace for Lossless Distributed
Video Coder
19
Rate-PSNR Curve for Lossy Distributed Video Coder
20
Conclusion

• Our coders achieve rates close to oracle
• Better than no estimation
• Motion estimation is more effective for more bits
  and considerable motion
• Better than intra-coding for lossless case and
  most of lossy cases

21

• Questions?

22
AppendixSequence Rate Trace (7 bits)
23
AppendixSequence Rate Trace (6 bits)
24
AppendixSequence Rate Trace (5 bits)