Investigation of MotionCompensated Lifted Wavelet Transforms

1
Investigation of Motion-Compensated Lifted
Wavelet Transforms
Markus Flierl and Bernd Girod
Information Systems Laboratory Department of
Electrical Engineering Stanford University
2
Outline
Can motion-compensated wavelet coding really do
better than motion-compensated predictive coding?
Why?

• Motion-compensated wavelet coding scheme
• Experimental results for temporal Haar and 5/3
  wavelets
• Mathematical model and performance bounds
• Comparison to predictive coding

3
Motion-Compensated Wavelet Coder
Original frames
H
H
LH
LH
LLH
LLL
Temporal Decomposition
Intraframe Coder

• DWT/ MC-Lifting
• Haar and 5/3 wavelet
• 16x16 block motion compensation
• ½ pixel accuracy

• 8x8 DCT coder
• Run-level entropy coding
• Same quantizer step-size in all frames

4
Motion-Compensated Haar Wavelet
Low
Even frames
Odd frames
High
Update step uses negative motion vector of
corresponding prediction step
5
Motion-Compensated 5/3 Wavelet
Frame 0
Low
Frame 1
High
Frame 2
Low
Update steps uses negative motion vectors of
corresponding prediction steps
6
R-D Performance of M.C.Wavelet Coder
Mother Daughter, QCIF, 30 fps

7
R-D Performance of M.C. Wavelet Coder
Mobile Calendar, QCIF, 30 fps
8
Mathematical Model

• Can we explain the experimental findings by a
  mathematical model? Yes!
• Extend rate-distortion analysis of
  motion-compensated hybrid coding to
  motion-compensated subband coding
• B. Girod, "Efficiency Analysis of
  Multi-Hypothesis Motion-Compensated Prediction
  for Video Coding," IEEE Trans. Image Processing,
  vol. 9, no. 2, pp. 173-183, February 2000.  
• B. Girod, "Motion-Compensating Prediction with
  Fractional-Pel Accuracy," IEEE Transactions on
  Communications, vol. 41, no. 4, pp. 604-612,
  April 1993.
•  
• B. Girod, "The Efficiency of Motion-compensating
  Prediction for Hybrid Coding of Video Sequences,"
  IEEE Journal on Selected Areas in Communications,
  vol. SAC-5, no. 7, pp. 1140-1154, August 1987.

9
Equivalent Motion-Compensated Haar Transform
Assume invertible motion compensation operations
Low
Even frames
Odd frames
High
10
Equivalent Motion-Compensated Haar Transform
11
Mathematical Model of Motion-Compensated
Transform
Any input picture can be reference picture
12
Coding Gain of Motion-Compensated Transform

• Rate difference for each picture k
• Difference of rate-distortion functions at high
  rates
• Compares m.c. transform coding to independent
  coding of frames
• . . . for the same mean squared reconstruction
  error
• . . . for Gaussian signals
• Total rate difference

13
Rate Difference with Negligible Noise
Calibration ? 0.5 log2(12 ?2?)
14
Rate Difference with White Noise
Calibration ? 0.5 log2(12 ?2?)
White noise at -30 dB
15
Comparison to Predictive Coding

• Predictive coding scheme
• Motion-compensated hybrid coder (like MPEG,
  H.263, . . . )
• 16x16 block motion compensation with half-pel
  accuracy
• Previous reference frame only
• Intra-frame coding with 8x8 DCT and run-length
  coding
• Only one I-frame in the beginning of the sequence
• Same quantizer step-size for all P-frames
• Same components as motion-compensated wavelet
  coding scheme

16
Comparison to Predictive Coding
Mother Daughter, QCIF, 30 fps
17
Comparison to Predictive Coding
Mobile Calendar, QCIF, 30 fps
18
Comparison to Predictive Coding
Calibration ? 0.5 log2(12 ?2?)
19
Conclusions

• Investigated motion-compensated wavelet transform
  followed by intra-frame coder both experimentally
  and theoretically
• Biorthogonal 5/3 wavelet outperforms Haar wavelet
• Wavelet transform can outperform predictive
  coding with single reference frame
• Theory offers insights and some possible
  explanations
• Rate can decrease up to 1 bpp per displacement
  accuracy step
• Gain by accurate motion compensation is limited
  by residual noise
• Motion-compensated transform can outperform
  predictive coding by up to 0.5 bpp, due to better
  noise suppression
• Long GOPs needed for wavelet subband decomposition