MCTF in Current Scalable Video Coding Schemes

1

• MCTF in Current Scalable Video Coding Schemes

Student Chia-Yang TsaiAdvisor Prof. Hsueh-Ming
HangInstitute of Electronics, NCTU
2
Outline

• Overview
• MCTF in Interframe Wavelet
• MCTF in JSVM
• Comparison
• References

3
Outline

• Overview
• Scalable Video Coding
• MCTF in Interframe Wavelet
• MCTF in JSVM
• Comparison
• References

4
Scalable Video Coding

• Ability to adjust
• Different client requirements
• Scalabilities
• Rate/SNR
• Spatial
• Temporal

5
MCTF
MCTF Motion Compensated Temporal Filtering
6
Rate/SNR Scalability

• Progressive approximation

300kbps PSNR32.2 dB
500kbps PSNR34.6 dB
1000kbps PSNR38.2 dB
GOP Header
Motion Info.
Image Data
7
Spatial Scalability

• Wavelet decomposition provides spatial scalability

8
Temporal Scalability
30Hz Video Sequence
15Hz Video Sequence
H
H
H
H
H
H
H
H1
7.5Hz Video Sequence
H2
H2
H2
H2
3.25Hz Video Sequence
H3
H3
L
H4
9
Scalable Video Coding

• History

HHI (AVC-based) ?JSVM
UNSW (wavelet)
RPI (wavelet)
MSRA (wavelet)
2005
2004.3
2004.7
10
Approaches
An AVC/H.264-based approach (also DCT-based)
11
Approaches
A wavelet-based approach with t2D structure.
12
Approaches
A wavelet-based approach with 2Dt structure
13
Lifting Scheme

• 5/3 lifting scheme

14
Outline

• Overview
• MCTF in Interframe Wavelet
• Barbell lifting
• In-band MCTF
• Base-layer structure
• MCTF in JSVM
• Comparison
• References

15
Barbell Lifting Scheme

• Purpose
• Improve the accuracy of motion field.
• Methods
• Take (5,3) wavelet kernel.
• Use barbell function to generate prediction
  /update values.

16
Barbell Lifting Scheme
17
Barbell Lifting Scheme
Prediction Stage
Update Stage
18
In-Band MCTF

• Purpose
• Improve coding performance with spatial
  scalability
• Methods
• Leaky motion compensation
• Mode-based temporal filtering

19
In-Band MCTF

• The forming of different quality reference of LL
• Low quality reference as IP_DIR

• High quality reference as IP_LBS

20
In-Band MCTF

• Leaky motion compensation
• leaky factor
• Attenuate the prediction based on the unknown
  information at the decoder
• make a good trade-off between drifting errors and
  coding efficiency

21
In-Band MCTF

• Mode-based temporal filtering
• Mode I Low quality reference
• Mode 2 High quality reference
• Mode is selected by RD cost

22
Base-Layer Structure

• Purpose
• Coding efficiency improvement in low rates
• AVC compatible
• Methods
• Insert AVC encoding module into MCTF

23
Base-Layer Structure
Encoder
Decoder
24
Outline

• Overview
• MCTF in Interframe Wavelet
• MCTF in JSVM
• Base layer structure
• Inter-layer prediction
• Adaptive prediction/update steps
• Comparison
• References

25
Base Layer Structure

• Purpose
• Coding efficiency improvement in low rates
• Compatibility to AVC
• Methods
• Unrestricted MCTF (UMCTF)
• Hierarchical B pictures

26
Base Layer Structure

• UMCTF
• Update step is omitted.
• Hierarchical B pictures
• Fully compatible to AVC Main profile
• Non-dyadic decomposition is available

27
Non-Dyadic Decomposition
28
Inter-Layer Prediction

• Purpose
• Reduce redundancy between layers
• Methods
• Inter-layer texture prediction
• Inter-layer motion prediction

29
Inter-Layer Prediction
30
Adaptive Prediction/Update Steps

• Purpose
• Delay (Memory)control
• Method
• Sub-partitioning of GOP

31
Adaptive Prediction/Update Steps
32
Outline

• Overview
• MCTF in Interframe Wavelet video
• MCTF in JSVM
• Comparison
• Cons and pros
• Experimental results
• References

33
Wavelet Based SVC

• Key features
• 3D wavelet decomposition
• Open-loop prediction structure
• Spatial-temporal resolution scalability
• SNR scalability

34
Wavelet Based SVC

• Advantages
• Nature for multi-resolution scalability
• Open-loop prediction structure
• Provides elegant SNR scalability without
  impairing full exploitation of spatial-temporal
  correlation
• Simplifies the R-D model of the bitstreams.
• Facilitates the bitstream truncation
• each subband is independent with other subbands

35
Wavelet Based SVC

• Disadvantages
• Decomposition modes (coding modes) selection
• Texture side information trade off
• Intra-prediction
• Badly-matched blocks
• Downsampling filter problems

36
AVC Based SVC

• Key features
• MCTF/Hierarchical B structure for temporal
  scalability
• Hierarchical B structure with close-loop
  structure for base layer
• Multiple spatial layers for spatial scalability
• Multiple FGS layers at each spatial resolution
  for SNR scalability
• DCT coding of all the frames

37
AVC Based SVC

• Advantages
• All the RDO and intra-prediction can be used.
• It guarantees the quality of the first testing
  point.
• MPEG filter for low resolution video
• the target low resolution video is visually
  good.

38
AVC Based SVC

• Disadvantages
• Redundancy between spatial layers

39
Experiments
40
Experiments
41
References

• 1 Draft of joint scalable video model
  (JSVM)3.0 reference encoding algorithm
  description, ISO/IEC JTC1/SC29/WG11, N7311,
  Poznan, July 2005.
• 2 D. Zhang, J. Xu, H. Xiong, and F. Wu,
  Improvement for in-band video coding with
  spatial scalability, ISO/IEC JTC1/SC29/WG11,
  M11681, HongKong, Jan. 2005.
• 3 V. Bottreau, G. Pau, and J. Xu, Vidwav
  evaluation software manual, ISO/IEC
  JTC1/SC29/WG11, M12176, Poznan, July. 2005.
• 4 X. Ji, J. Xu, D. Zhao, and F. Wu, Responses
  of CE1d base- layer, ISO/IEC JTC1/SC29/WG11,
  M11127, Redmond, July 2004.
• 5 R. Xiong, J. Xu, and F. Wu, Coding
  performance comparison between MSRA wavelet video
  coding and JSVM, ISO/IEC JTC1/SC29/WG11, M11975,
  Busan, April 2005.
• 6 R. Xiong, J. Xu, and F. Wu, Response to
  VidWav EE1, ISO/IEC JTC1/SC29/WG11, M12286,
  Poznan, July 2005.
• 7 J. Reichel, K. Hanke and B. Popescu,
  Scalable Video Model V1.0, ISO/IEC
  JTC1/SC29/WG11, N6372, Munich, March 2004.