H.264/Advanced Video Coding

1
H.264/Advanced Video Coding A New Standard

• Song Jiqiang
• Oct 21, 2003

2
The Scope of Video Coding Standardization

• Only restrictions on the Bitstream, Syntax, and
  Decoder are standardized
• Permits the optimization of encoding
• Permits complexity reduction for implementability
• Provides no guarantees on quality

3
Chronological Table of Video Coding Standards
ITU-T VCEG
H.263 (1995/96)
H.263 (2000)
H.263 (1997/98)
H.261 (1990)
H.264 ( MPEG-4 Part 10 ) (2003)
MPEG-2 (H.262) (1994/95)
MPEG-4 v1 (1998/99)
ISO/IEC MPEG
MPEG-4 v2 (1999/00)
MPEG-1 (1993)
MPEG-4 v3 (2001)
1990 1992 1994 1996 1998
2000 2002 2003
4
H.261 The Basis of Modern Video Compression

• The first widespread practical success
• Milestone The first design establishes the
  typical hybrid structure that dominates today.
• 16?16 marcoblock motion compensation
• 8?8 DCT transform
• Variable-length entropy coding
• Operated at 64-2048 Kbps (p?64Kbps)
• Still in use
• Low complexity, low latency
• Mostly as a backward-compatibility feature
• Overtaken by H.263

5
MPEG-1 For Storage

• Five parts System, Visual, Audio, Conformance,
  Reference Software
• Applications VCD, VOD, Digital Camera
• Superior quality to H.261 when operated at higher
  bit rates ( 1 Mbps for CIF 352x288 resolution)
• Provides approximately VHS quality between 1-2
  Mbps using SIF 352x240/288 resolution
• Technical features Adds bi-directional motion
  prediction and half-pixel motion to H.261 design
• Use is fairly widespread, but mostly overtaken by
  MPEG-2

6
MPEG-2 / H.262 High Bit Rate, High Quality

• MPEG-2 contains 10 parts
• MPEG-2 Visual H.262
• Not especially useful below 2 Mbps (range of use
  normally 2-20 Mbps)
• Applications SDTV (2-5Mbps), DVD (6-8Mbps), HDTV
  (20Mbps), VOD
• Support for interlaced scan pictures
• PSNR, temporal, and spatial scalability
• Profile and Level
• 10-bit precision video sampling

7
H.263 The Next Generation

• Has overtaken H.261 as dominant
  video-conferencing codec
• Superior to H.261 at all bit rates
• Wins by a factor of two at very low rates
• Four basic options UMV, SAC, Advanced prediction
  mode, PB-frame
• H.263 (1998) supports all bit rates, more
  options
• H.263 (2000) more options, emphasizing on
  error resilience and scalability

8
MPEG-4 H.263 Additions Variable Shape Coding

• Goal Support for interactive multimedia
• Visual Object (AO), Audio Object (AO) and AVO
• Roughly follows H.263 design and adds all prior
  features and (most important) shape coding
• 18 video coding profiles
• Includes zero-tree wavelet coding of still
  textured pictures, segmented coding of shapes,
  coding of synthetic content
• 2D 3D mesh coding, face animation modeling
• 10-bit and 12-bit video
• Contains 9 parts. Part 10 will be H.264

9
A Note on Terminology of H.264

• The following terms are used interchangeably
• H.26L
• The Work of the JVT or JVT CODEC
• JM2.x, JM3.x, JM4.x
• The Thing Beyond H.26L
• The AVC or Advanced Video CODEC
• Proper Terminology going forward
• MPEG-4 Part 10 (Official MPEG Term)
• ISO/IEC 14496-10 AVC
• H.264 (Official ITU Term)

10
Position of H.264
11
New Features of H.264

• Multi-mode, multi-reference MC
• Motion vector can point out of image border
• 1/4-, 1/8-pixel motion vector precision
• B-frame prediction weighting
• 4?4 integer transform
• Multi-mode intra-prediction
• In-loop de-blocking filter
• UVLC (Uniform Variable Length Coding)
• NAL (Network Abstraction Layer)
• SP-slices

12
Profiles and Levels

• Profiles Baseline, Main, and X
• Baseline Progressive, Videoconferencing
  Wireless
• Main esp. Broadcast
• X Mobile network
• Baseline profile is the minimum implementation
• Without CABAC, 1/8 MC, B-frame, SP-slices
• 11 levels
• Resolution, capability, bit rate, buffer,
  reference
• Built to match popular international production
  and emission formats
• From QCIF to D-Cinema

13
Basic Marcoblock Coding Structure
14
Motion Compensation
15
Multiple Reference Frames
16
B-frame Prediction Weighting

• Playback order I0 B1 B2 B3 P4 B5
  B6 ...
• Bitstream order I0 P4 B1 B3 B2 P8
  B5 ...

17
4?4 Integer Transform
18
Intra-prediction Modes
19
In-loop De-blocking Filter

• Highly compressed decoded inter picture
• Significantly reduces prediction residuals

Without filter with H.264/AVC
De-blocking
20
SP-Slices

• Efficiently switching between two bitstreams
• Provides VCR-like functions

21
Comparison
22
Comparison
23
Summary

• Video coding is based on hybrid video coding and
  similar in spirit to other standards but with
  important differences
• New key features are
• Enhanced motion compensation
• Small blocks for transform coding
• Improved de-blocking filter
• Enhanced entropy coding
• Substantial bit-rate savings (up to 50) relative
  to other standards for the same quality
• Enhancement on visual quality seems better than
  that on PSNR
• The complexity of the encoder triples that of the
  prior ones
• The complexity of the decoder doubles that of the
  prior ones

24
Implementations and Implementability

• UB Video (JVT-C148) CIF resolution on 800 MHz
  laptop
• Encode 49 fps, Decode 137 fps
• EncodeDecode 36 fps
• Better quality than R-D optimized H.263 Profile
  3 (IJKT) while using 25 higher rate and
  low-delay rate control
• Videolocus (JVT-D023) SDTV resolution
• 30 fps encode on P4 2 GHz with hardware assist
• Decode on P3 1 GHz laptop (no hardware assist)
• No B frames, no CABAC (approx baseline)
• DGFX SDTV, HDTV SW Encoders, Decoders and
  PreProcessing
• 10-12 Bit Implementation
• Others HHI, Deutsche Telekom, Broadcom, Nokia,
  Motorola, NDS/Tandberg, Harmonic, LSI Logic, etc.

25
Questions?Thank you!