Transcoding of an MPEG2 bit stream to an H.264 bit stream

1
Transcoding of an MPEG-2 bit stream to an H.264
bit stream

2
What is Transcoding ?

• The operation of converting video in one format
  to another format.
• Need Compatibility between MPEG-2 and H.264
  devices
• Applications To adapt the bit rate of a
  compressed stream to the channel bandwidth, to
  change the spatial or temporal resolution of a
  compressed stream etc.

3
Criteria considered in Heterogeneous Transcoding

• Quality of the transcoded stream should be
  comparable to that obtained by complete decoding
  and re-encoding with full motion search and to
  that of the initial input stream.
• The information in the input bit stream should be
  re-used as much as possible to reduce
  multigenerational degradation.
• The computational cost and complexity should be
  kept minimal.

4
MPEG-2 Decoder
MPEG-2 bit stream
5
H.264 encoder
6
Transcoding Algorithm

7
Intra frame coding

8
Intra Frame Transcoding
9
Complexity of applying mode decisions in
transform domain

• Example Vertical prediction

Predicted block
10
Intra modes in H.264/AVC
Directional modes for an intra 4x4 macroblock
Directional prediction
Directional modes for an intra 16x16 macroblock
11
Mode decision algorithm
12
Why use standard deviation?

• Simple metric
• Can be easily computed as the transform domain
  coefficients are already available.

13
Post mode decision
14
Intra Frame transcoding results
15
Subjective Quality of Intra framesMPEG-2 Input
Stream H.264 bit stream obtained
H.264 bit stream Test clip Akiyo
by the proposed method obtained from
complete Bit rate 1
Mbps Bit rate 768 Kbps decoding and
re-encoding Spatial resolution 352x240
of the input MPEG-2 bit stream

Bit
rate 670 Kbps
16
Subjective Quality of Intra framesMPEG-2 Input
H.264 bit stream obtained
H.264 bit stream bit stream by the
proposed method obtained from complete
Test clip Foreman Bit rate 1Mbps
decoding and re-encoding Bit rate 1 Mbps
of the input MPEG-2 bit Spatial resolution
352x240 stream. Bit rate 1Mbps

17
Inter frame coding
18
Inter frame transcoding
MPEG-2 bit stream
Inverse quantise
Sum residuals
VLD
IDCT
Hierarchical mode decisions
MV Refinement
Pass parameters
Inter prediction
Rate control
H.264 bit stream
4x4 Integer transform
Quantise
VLC/CABAC
Inverse VLC/CABAC
Inverse Quantise
Motion compensate
Store as reference frame
19
Inter frame transcoding

• Features
• Motion vector extraction
• Motion vector refinement
• Motion vector reuse
• Hierarchical mode decision

20
Motion vector extraction

• Motion vectors can be extracted from the MPEG-2
  bit stream after variable length decoding.

21
Need for motion vector refinement

• Need
• Differences in the quantization parameters of the
  incoming bit stream and those selected may
  differ. When these differences are large it
  results in quality degradation.
• MPEG-2 supports certain modes in which no motion
  information is coded. However, since H.264
  supports more fine motion estimation block sizes,
  a small amount of motion may result upon
  refinement.
• Re-evaluation of the decision to intra code
  macroblocks in a P frame.
• Improves accuracy of the motion vectors and helps
  achieve compatibility between ½ pel MV accuracy
  in MPEG-2 and ¼ pel MV accuracy in H.264

22
Need for motion vector refinement

• Compensates for field coding to frame coding
  changes and vice versa

23
Motion vector refinement

• MPEG-2 motion vectors are refined over a one
  pixel window i.e. dx dy 3 pixels , in the
  most recent reference frame in List 0.
• Half pixel and quarter pixel refinement is
  performed with the defined window.

24
Search window size (dx,dy) selection

• Before window size selection, different
  increasing window sizes were tested to verify the
  effect of varying the search window size on the
  PSNR.
• The graph for one such test clip Akiyo is as
  follows
• It was observed that the PSNR obtained for a one
  pixel window closely approximated the steady
  state value and using a one pixel window provided
  a good tradeoff between complexity and the PSNR.

25
Motion vector reuse
26
Hierarchical mode decisions

• Coding modes are compared and selected based on
  the sum of absolute difference (SAD) value. In
  the full mode decision method, every coding mode
  is evaluated ,the SAD value is computed and the
  mode with the minimum SAD value is selected as
  the best mode. However , although this method
  would give the best results ,it is very
  computationally intensive. For instance, each
  macroblock in the P frame would have to be
  evaluated for 16x16, 2 16x8, 2 8x16, 4 8x8, 8
  8x4,8 4x8, 16 4x4 intra and skip modes
• Hierarchical mode decision process makes use of
  the fact that after evaluating a mode and the
  next level of sub partitioned modes , if sub
  partitioning does not reduce the SAD value then
  further sub partitioning need not be evaluated.

27
Hierarchical mode decisions

• The top down splitting approach is shown below

28
P frame transcoding Results

29
P frame results

(see previous slide)
30
P frame results
31
Motion vectors in P framesMPEG-2 Input Stream
H.264 bit stream obtained H.264
bit stream Test clip Akiyo by
the proposed method obtained from
complete Bit rate 1 Mbps Bit rate 768
Kbps decoding and re-encoding Spatial
Resolution352x240 of the input MPEG-2 bit
stream

Bit rate 670 Kbps
H.264 motion vectors after transcoding
MPEG-2 motion vectors
H.264 motion vectors after full motion search
32
Mode decisions in P framesMPEG-2 Input Stream
H.264 bit stream obtained H.264
bit stream bit stream. by
the proposed method obtained from
complete Test clip Akiyo Bit rate 768
Kbps decoding and re- Bit rate 1 Mbps
encoding of the input Spatial
resolution352x240 MPEG-2 bit stream

Bit rate
670 Kbps
16x16 modes in the MPEG-2 bit stream
16x16 and sub-macroblock modes in the H.264
transcoded bit stream
16x16 and sub-macroblock modes in the re-encoded
H.264 bit stream
33
B frame transcoding results

34
B frame transcoding results
35
B frame transcoding results

36
Motion vectors in B framesMPEG-2 Input Stream
H.264 bit stream obtained H.264
bit stream. Test clip Akiyo
by the proposed method obtained
from complete Bit rate
1 Mbps Bit rate 768 Kbps decoding and
re-encoding Spatial resolution352x240
of the input MPEG-2 bit stream

Bit
rate 670 Kbps
Backward motion vectors in the H.264 transcoded
bit stream
Forward motion vectors in MPEG-2
Forward motion vectors in the H.264
transcoded bit stream
Forward motion vectors in the H.264 transcoded
bit stream
Backward motion vectors in the H.264 transcoded
bit stream
Backward motion vectors in MPEG-2
37
Mode decisions in B framesMPEG-2 Input Stream
H.264 bit stream obtained H.264
bit stream bit stream. by
the proposed method obtained from
complete Test clip Akiyo Bit rate 768
Kbps decoding and re-encoding Bit rate 1 Mbps
of the input MPEG-2 bit Spatial
resolution352x240 stream

Bit rate 670 Kbps
16x16 and sub 16x16 modes in the H.264
transcoded bit stream
16x16 and sub 16x16 modes in the H.264
re-encoded bit stream
16x16 modes in the MPEG-2 bit stream
38
B frame transcoding results
39
Comparison of the Input MPEG-2 bit stream vs. the
transcoded H.264 bit stream

• The table below illustrates the comparison
  between the PSNR of the input MPEG-2 bit stream
  and the PSNR of the transcoded H.264 bit stream
  obtained by transcoding 35 frames at 1Mbps with
  the IBBPBBP GOP structure

40
Comparison of the proposed method with the DCT
domain transcoder proposed by Chang and
Messerschmitt 23

• The graph shown compares the proposed method with
  DCT domain transcoding 23 and complete decoding
  and re-encoding of a 1 Mbps MPEG-2 bit stream
  (test clip Foreman) to an H.264 bit stream with
  an IBBPBBP.GOP structure at a constant bit rate.

41
Proposed method transcoded stream

• Proposed method
• Full re-encoding

42
References

• 1 J. Youn and M-T. Sun , Motion Vector
  Refinement for high-performance transcoding, in
  IEEE Int. Conf. Consumer Electronics, Los
  Angeles, CA, Vol. 1, Issue 1, pp. 30-40, March
  1999.
• 2 J. Xin, C-W. Lin and M-T. Sun, Digital Video
  Transcoding , Proceedings of the IEEE, Vol. 93,
  pp. 84-97, Jan. 2005.
• 3 T. Wiegand et. al., Overview of the
  H.264/AVC Video Coding Standard, IEEE Trans.
  CSVT, Vol. 13, pp. 560-576, July 2003.
• 4 A. Vetros, C. Christopoulos and H. Sun,
  Video transcoding architectures and techniques
  an overview, IEEE Signal Processing magazine,
  Vol. 20, pp. 18-29,March 2003.
• 5 H. Kalva, Issues in H.264/MPEG-2 Video
  Transcoding, IEEE Consumer Communications and
  Networking Conf., CCNC 2004, pp 657-659, Jan
  2004.
• 6 Information Technology-Generic coding of
  moving pictures and associated audio information
  Video, ITU-T Rec. H.262 (2000 E).
• 7 B. Haskell, A. Puri and A. Netravali,
  Digital Video an introduction to MPEG-2, N.Y.
  Chapman and Hall, International Thomson Pub.,
  1997.
• 8 G. Chen et. al., Efficient block size
  selection for MPEG-2 to H.264 transcoding,
  Proceedings of the 12th annual ACM International
  Conference on Multimedia, pp. 300-303, Oct. 2004.
• 9 MPEG-2 software (version 12) from MPEG
  software simulation group, http//www.mpeg.org/MPE
  G/MSSG/source
• 10 H.264 Software (JM9.5) from
  http//iphome.hhi.de/suehring/tml/download/jm94.zi
  p
• 11 A. Puri, X. Chen and A. Luthra, Video
  coding using the H.264/MPEG-4 AVC compression
  standard, Signal processing Image
  communication, Vol. 19, pp. 793-849, Oct. 2004.
• 12 B. Shen and I. Sethi, Direct feature
  extraction from compressed images, SPIE Vol.
  2670 Storage and Retrieval for Image Databases
  IV, pp. 404-414, 1996.
• 13 Commercially available transcoders, PSP
  Video 9, http//www.pspvideo9.com
• 14 K.R. Rao and J. J. Hwang, Techniques and
  Standards for Image, Video and Audio coding,
  Upper Saddle River, N.J. Prentice Hall, 1996.

43
References continued

• 15 M. Ghanbari, Video Coding an introduction
  to standard codecs, London, U.K. Institution of
  Electrical Engineers, 1999.
• 16 I. E. G. Richardson, H.264 and MPEG-4
  video compression video coding for next
  generation multimedia, Chichester Wiley, 2003.
• 17Test streams obtained from ftp//ftp.tek.com/t
  v/test/streams/Element/MPEG-Video/525/ and
  http//www.cipr.rpi.edu/resource/sequences/sif.htm
  l
• 18 Y-J. Chuang, Y-C. Huang and J-L Wu, An
  efficient block algorithm for splitting an 8x8
  DCT into four 4x4 modified DCT used in
  AVC/H.264, EURASIP 2005, pp. 311-316.
• 19 P. Assunco and M. Ghanbari, Post Processing
  of MPEG-2 coded video for transmission at lower
  bit rates, Proc. IEEE ICASSP, pp. 1998-2001,
  Atlanta, GA, 1996.
• 20 T. Shanableh and M. Ghanbari, Transcoding
  Architectures for DCT domain heterogeneous video
  transcoding, Proc. IEEE ICIP, Vol. 1, pp.
  433-436, Thessaloniki, Greece, Sept. 2001,.
• 21 J. Xin, M.T. Sun and K. Chun, Motion
  re-estimation for MPEG-2 to MPEG-4 simple profile
  transcoding, Proc. Int. Workshop Packet Video,
  Pittsburgh, PA, Apr. 2002.
• 22 D-Y. Chan, S-J. Lin and C-Y. Chang, A rate
  control scheme using Kalman filtering for H.263,
  Journal of Visual Communication and Image
  Representation, Vol. 16, pp. 734-748, Dec. 2005.
• 23 S. Liu and A. Bovik, Foveated embedded DCT
  domain video transcoding, Journal of Visual
  Communication and Image Representation, Vol. 16,
  pp. 643-667, Dec. 2005.
• 24 I. E. G. Richardson, Video codec design
  developing image and video compression systems,
  Chichester Wiley, 2002.
• 25 G. Sullivan, T. Wiegand and A. Luthra,
  Draft of Version 4 of H.264/AVC (ITU-T
  Recommendation H.264 and ISO/IEC 14496-10 (MPEG-4
  part 10) Advanced Video Coding), JVT Doc., 14th
  Meeting Hong Kong, China 18-21 Jan. 2005.
• 26 G. F-Escribano et.al., Computational
  complexity reduction of intra frame prediction in
  MPEG2/H.264 video transcoders, ICME, pp.
  707-710, July 2005.

44
References continued

• 27 I. Ahmad et. al., Video transcoding an
  overview of various techniques and research
  issues, IEEE Trans. on multimedia, vol. 7, pp.
  793-804, Oct. 2005.
• 28 S. Benyaminovich, O. Hadar and E. Kaminsky,
  Optimal transrating via DCT coefficients
  modification and dropping, ITRE, pp. 100-104,
  June 2005.
• 29 J-R. Ohm, Advances in scalable video
  coding, Proc. IEEE, Vol. 93, pp. 42-56, Jan.
  2005.
• 30 J. Wang et. al., An AVS to MPEG-2
  transcoding system, Proc. of ISIMP, pp.302-305,
  Oct. 2004.
• 31 J. McVeigh et. al., A software based
  real-time MPEG-2 video encoder, IEEE Trans.
  CSVT, Vol. 10, pp. 1178-1184, Oct. 2000.