H'264

1
H.264

• Subhrendu Sarkar
• Computer Science,
• Columbia University
• COMS W4995 - VOIP Security

2

• Introduction
• Video Formats and Quality
• Video Coding and H.264
• Performance
• Conclusion
• References

3
Introduction

• What is H.264 ?
• H.264 is a Video Coding Standard also known as
  MPEG-4 Part-10 (AVC).
• H.261, H.263, MPEG-1, MPEG-2 are some
  predecessors of H.264.
• Purpose of a standard
• Define a coded representation (or syntax) that
  describes visual data in a compressed form and
  method of decoding the syntax to reconstruct
  visual information.
• compliant encoders and decoders can successfully
  interoperate with each other.
• A Video standard specifically do not define an
  encoder rather, they define the output that an
  encoder should produce.
• A decoding method is defined in each standard.

Video from Latin I see
4

• Introduction
• Video Formats and Quality

5

• Pixel (Picture Element)
• Interlaced Video (Frames and Fields)
• Bitrate and Frame rate.
• Typically 30 fps is good for human visual system.
• Higher the bitrate, better the quality of video.
• Higher frame rate, better the quality of video

6

• Video Formats
• NTSC , PAL (analogue video)
• PAL (Europe, Asia, Australia, etc.) 25 frames/sec
• SECAM (France, Russia, parts of Africa etc.) 25
  frames/sec
• NTSC (USA, Canada, Japan, etc.) 29.97 frames/sec
  
• According to resolution
• VGA 640x480 (Video Graphics Array), QVGA
• CIF 352x288 CIF (Common Intermediate Format)
• QCIF, SQCIF
• SDTV (e.g 720 x 480)
• HDTV (e.g 19201080 )
• Color Spaces
• RGB (Red, Green, Blue)
• YUV also known as YCbCr (luminance, chroma)
• Y kr R kgG kbB

7

• YUV Sampling Formats
• YUV 444
• YUY2 (422)
• YV12 or YUV420 (420)

Courtesy Images from H.264 and MPEG-4
Compression Ian Richardson
8
Video Quality

• Subjective Video Quality
• Objective Video Quality
• PSNR (Peak Signal to Noise Ratio)
• measured on a logarithmic scale and depends on
  the mean
• squared error (MSE) of between an original and an
• impaired Image or video frame, relative to (2n
  -1)2 (the
• square of the highest possible signal value in
  the image,
• where n is the number of bits per image sample).

9

• Introduction
• Video Formats and Quality
• Video Coding and H.264

10

• A video CODEC encodes a source image or video
  sequence into a compressed form and decodes this
  to produce a copy or approximation of the source
  sequence.

• Three main functional units of a Video Encoder
• Spatial Model
• Temporal Model
• Entropy Encoder

Courtesy Images from H.264 and MPEG-4
Compression Ian Richardson
11

• Macroblock, Block and Sub-Block.
• 16 x16 Macroblocks.
• 16 x 8, 8 x16, 8x8, 8x4, 4x8, 4x4 Blocks.
• Temporal Model
• Prediction from the Previous Video Frame
• Optical Field Flow
• Block based Motion Estimation and Motion
  Compensation

Ref Block
Ref Block
Current Macroblock
12
Motion vectors

• Search region, finds best matching MB
• mv mvx and mvy (distance between current block
  and the ref macroblock.
• mvp - motion vector predictor
• mvd - motion vector difference
• mvd Difference (mvp, mv)

B mvp
D mvp
C mvp
A mvp
mvd
Courtesy Diagram from http//wiki.multimedia.cx
/index.php?titleMotion_Prediction
13
16x16 Motion Vectors
Frame Fn
Frame Fn-1
Motion Compensated Residual
Motion Compensated Reference
Residual Fn Fn-1
Courtesy Images from H.264 and MPEG-4
Compression Ian Richardson
14

• I Frames, P Frames, B Frames
• I Frames Spatial prediction only for all MBs
• P Frames has I (intra) spatially predicted MB
  and P (inter) temporally predicted MBs.
• B Frames Bi-directionally predicted frames.
• Group of Pictures (GOP)
• Display Order
• Frame No 0 1 2 3 4 5 6 7 8
  9
• Frame Type I B B P B B P B B
  I ...
• Encoding of Decoding Order
• Frame No 0 3 1 2 6 4 5
  7 8 9
• Frame Type I P B B P B B B
  B I ...

15

• Spatial Model
• Intra Macroblocks.
• Spatial Correlation between Macroblocks
• Temporal Model
• Inter macroblocks
• Temporal Correlation between Macroblocks
• Searching for similar macroblocks from reference
  frames.
• Transform
• Time to frequency domain.
• Discrete Cosine Transform is used.
• Theoretically not lossy.

Courtesy Diagrams from H.264 and MPEG-4
Compression Ian Richardson
16

• Quantization
• Basically dividing the transformed coefficients
  by quant values in the encoder and multiplying
  by the quant value in the decoder.
• Lossy
• Helps meet bitrate constraints.
• Human eye is less sensitive to higher frequency
  transform coefficients.
• Entropy Coder
• Reorder (Zig-Zag Scan order)
• Variable length coding
• Run Length Coding
• Huffman Coding
• Arithmetic Coding
• H.264
• Context Adaptive Variable Length Coding. (CAVLC)
• Context Adaptive Binary Arithmetic Coding.
  (CABAC)

17
Video Encoder
In Loop Deblocking Filter
Courtesy Diagrams from H.264 and MPEG-4
Compression Ian Richardson
18
Video Decoder
Deblocking Filter
Courtesy Diagrams from H.264 and MPEG-4
Compression Ian Richardson
19
Deblocking Filter
Non Deblocked Image
Deblocked Image
Courtesy Images from http//compression.ru/video
/deblocking/
20
H.264 Profiles
Courtesy Diagram from H.264 and MPEG-4
Compression Ian Richardson
21

• Introduction
• Video Formats and Quality
• Video Coding and H.264
• Performance

22
Sample Videos

• H.263
• MPEG-4 Basic
• MPEG-4 Improved
• MPEG-4 Part 10 (AVC) or H.264

QUALITY
Compression
Complexity
Courtesy Videos from http//mac.sillydog.org/qt/
compare.php
23
Applications
Courtesy Table from H.264 and MPEG-4
Compression Ian Richardson
24
Conclusion

• H.264 is a Digital Video Coding Standard
• H.264 gives best quality and compression when
  compared to earlier codec (s).
• H.264 more complex and thus requires more
  processing.
• Different Profiles cater to different kinds of
  applications.
• Tradeoff between Quality of Video, Compression
  achieved and complexity of the codec

25
References

• H.264 And MPEG-4 Video Compression
• Ian Richardson
• ISO/IEC 14496-10 and ITU-T Rec. H.264, Advanced
  Video Coding, 2003.
• H.264 Reference Software Version
• http//iphome.hhi.de/suehring/tml/
• Current software version JM 14.2

26
Appendix - H.264 Features

• Weighted Prediction
• Entropy Coding
• CAVLC (Context Adaptive Variable Length Coding)
• CABAC (Main profile) Context Adaptive Binary
  Arithmetic Coding

27
Some H.264 Features

• Reference Pictures
• Slices
• Integer and Sub-sample prediction

Courtesy Images from H.264 and MPEG-4
Compression Ian Richardson
28
Appendix - Rate Control
Courtesy Images from H.264 and MPEG-4
Compression Ian Richardson
29
Courtesy Images from H.264 and MPEG-4
Compression Ian Richardson