CHAPTER 8 MPEG4 Visual

1
CHAPTER 8MPEG-4 Visual
2
MPEG-4 Visual

• MPEG-1 and MPEG-2 deal with frame-based video and
  audio.
• The most important goal of these standards has
  been to make storage and transmission very
  efficient.
• The objective of MPEG-4 was to standardize
  algorithms for audiovisual coding in multimedia
  applications, allowing for interactivity, high
  compression, scalability of audio and video
  content, and support for natural and synthetic
  audio and video content.

3
MPEG-4 Architecture

• MPEG-4 defines an audiovisual scene as a coded
  representation of audiovisual objects that have
  certain relationships in space and time.
• This is different from MPEG-1 and MPEG-2, wherein
  the audiovisual scene is thought of as video
  frames with associated audio.
• This new approach to information representation
  offers much more flexibility for versatile reuse
  of data, intelligent schemes to manage bandwidth,
  processing resources, and error protection.

4
MPEG-4 Architecture (Cont.)

• The bit streams of each A/V object can be
  transmitted across multiple channels, where each
  channel offers a different quality of service.
• At the decoder. The compositor uses the
  spatio-temporal relationships and user
  interactions to render the scene.
• In the MPEG-4 notion of A/V objects, the object
  could be a video object within a scene or it
  could be the complete background. It could also
  be an audio object.

5
MPEG-4 Video Coding

• In MPEG-4, like MPEG-1 and MPEG-2, there is a
  hierarchical representation of the information.
  Each video frame is segmented into a number of
  arbitrary shaped image region, called video
  object planes (VOP).
• Successive VOPs belonging to the same physical
  object in a scene are referred to as video
  objects (VO).
• Since each VOP is coded separately, based on the
  decoded information from the alpha channel, the
  decoder can decode and display each VOP
  separately or reconstruct the original sequence
  by decoding and compositing both of them.
• MPEG-4 supports content-based scalability.

6
VOP
7
MPEG-4 Video Object Plane
8
MPEG-4 Video Object Plane
9
Video Object Plane

• The shape, motion, texture information of the
  VOPs belonging to the same VO is encoded into a
  separate video object layer (VOL).
• A binary alpha-plane-image sequence is coded to
  indicate to the decoder the shape of the
  foreground object and its location.
• Since each VOP is coded separately, based on the
  decoded information from the alpha channel, the
  decoder can decode and display each VOP
  separately or reconstruct the original sequence
  by decoding and compositing both of them.

10
Content-Based Coding of Video

• Object scalability and quality scalability
• MPEG-4 supports the overlapping configuration for
• VOPs as well.

11
Block Diagram of MPEG-4 Encoding and Decoding

• Each video object in a scene is coded and
  transmitted separately.

12
VOP-Based Encoding
13
Coding of Information for Each VOP

• For each VO, the shape, motion, and texture
  information of VOPs comprising this VO are coded.
• Shaping coding The shape information is referred
  to as alpha plane.
• Motion coding
• Temporal redundancies between video content in
  separate VOPs within a VO are exploited using
  block-based motion estimation and compensation.
• Macroblocks can be either standard or contour
  macroblocks.
• For contour macroblocks, prior to motion
  estimation and compensation of the current VOP in
  frame t, a simple image padding technique is used
  to reference the VOP of frame t-1.

14
Motion Compensation Tools
Motion compensated coding modes (I, B, P)
15
Motion Vector Computation

• Texture coding The intra VOPs, as well as
  residual errors after motion compensated
  prediction, are coded using DCT on 8 ? 8 blcoks.

16
Tools, Objects, and Levels

• MPEG-4 Visual provides its coding function
  through a combination of tools, objects, and
  profiles.
• A tool is a subset of coding functions to support
  a specific feature (e.g., basic video coding,
  interlaced video, coding object shapes, etc.)
• An object is a video element (e.g., a sequence of
  rectangular frames, a sequence of
  arbitrary-shaped regions, a still image) that is
  coded using one or more tools.
• A profile is a set of object types that a CODEC
  is expected to be capable of handling.

17
MPEG-4 Tools, Profiles, and Decoder Flexibility
18
Profiles and Levels
19
MPEG-4 Visual Profiles for Coding Natural Video
20
Levels for Simple-Based Profiles
21
VOP-Based Decoding
22
MPEG-4 Video Encoding
23
MPEG-4 Video Decoding
24
Computational Complexity of Video Encoder
25
A Fast Binary Motion Estimation Algorithm for
MPEG-4 Shape Coding

• Tsung-Han Tsai and Chia-Pin Chen
• Department of Enectronic Engineering
• National Central University

26
Introduction

• This paper presents a fast binary motion
  estimation (BME) algorithm using diamond search
  pattern for MPEG-4 shape coding, which is the key
  technology for supporting the content-based video
  coding.
• Based on the properties of binary shape
  information, a boundary mask for efficient search
  positions can be generated.
• Simulation results show that our algorithm
  combined with diamond shaped zone takes equal bit
  rate in the same but reduce the number of search
  points marvelously in BME to 0.6 compared with
  full search algorithm.

27
Texture Components and Binary Alpha Component of
the Video Object
28
Computational Complexity of MPEG-4 Shape Encoder
29
BME for MPEG-4 Shape Coding

• The MPEG-4 VM (verification model) describes the
  coding method for binary shape information. It
  uses block-matching motion estimation to find the
  minimum block distortion position and sets the
  position to be MVS (motion vector for shape).
• The procedure for BME consists of two steps
• Determine motion vector predictor for shape
  (MVPS)
• MVPS is taken from a list of candidate MVs.
• Compute MVS accordingly
• Fig. 3

30
BME for MPEG-4 Shape Coding (Cont.)

• The list of candidate MVs includes the MVSs from
  the three binary alpha blocks (BABs) which are
  adjacent to the current BAB and the texture MVs
  associated with the three adjacent texture
  blocks.
• Based on MVPS determined above, the motion
  compensated (MC) error is computed by comparing
  the BAB indicated by the MVPS and current BAB.
• If the computed MC error is less or equal to 16 ?
  AlphaTH for any 4 ? 4 subblocks, the MVPS is
  directly employed as MVS and the procedure
  terminates. Otherwise, MV is searched around the
  MVPS while computing by comparing the BAB
  indicated by the MV and current BAB. The MV that
  minimizes the SAD is taken as MVS and this is
  further interpreted as MV difference for shape
  (MVDS), i.e., MVDS MVS MVPS.

31
Proposed BME Method

• The basic concept of the proposed BME method is
  that the contour of video objects in current BAB
  should overlap that in the motion compensated
  BAB, which is determined by BME 9.
• Those search positions which contour lays apart
  from the contour of video objects in current BAB,
  can be skipped and the reduced number will be
  enormous.
• Moreover, based on the property that most
  real-world sequences have a central-biased MV
  distribution 10, we use weighted SAD and
  diamond search pattern for furthermost
  improvement.

32
Definition of Boundary Pixel

• In order to decide whether the pixel is on the
  contour of VOP, the boundary pixel is determined
  by the following procedure.
• If current pixel is opaque and one of its four
  adjacent pixels is transparent, the current pixel
  is directly employed as boundary

33
Boundary Search

• Step 1) Perform a pixel loop over the entire
  reference VOP. If pixel
• (x, y) is an boundary pixel, set the
  mask at (x, y) to 1.
• Otherwise set the mask at (x, y) to
  0.
• Step 2) Perform a pixel loop over the entire
  current BAB. If pixel (i, j)
• is a boundary pixel, set (i, j) to
  be the reference point, and
• terminate the pixel loop.

34
Boundary Search (Cont.)

• Step 3) For each search point within ?16 search
  range, check the
• pixel (xi, yj) which is fully
  aligned with the reference point
• from the current BAB. If the mask
  value at (xi, yj) is 1,
• which means that the reference point
  is on the boundary of
• reference VOP, the procedure will
  compute SAD of the
• search point (x, y). Otherwise, SAD
  of the search point (x, y)
• will not be computed, and the
  processing continues at the
• next position.
• Step 4) When all the search points within ?16
  search range is
• done, the MV that minimizes the SAD
  will be taken
• as MVS.

35
Boundary Search (Cont.)
36
Flowchart for Proposed BS Algorithm.
37
Diamond Boundary Search (DBS)

• Step 1) Construct DBS around
• MVPS within 16 search
• window. Set n 1.
• Step 2) Calculate SAD for each
• search point in zone n. Let
• MinSAD be the smallest
• SAD up to now.
• Step 3) If MinSAD ? Thn , go to Step
• 4. Otherwise, set n n 1
• and go to Step 2.
• Step 4) The MV is chosen according
• to the block corresponding to
• MinSAD.

5 search window using diamond-shaped zones.
38
Weighted SAD (WSAD)

• Some previous motion estimation algorithms for
  color space used the concept of the WSAD to
  compensate the distortion 12.
• In this paper, we proposed the similar concept of
  WSAD which takes both SAD and MVDS into
  consideration as the distortion measure. The WSAD
  is given by

39
Simulation Results
Performance comparisons of SAD and WSAD using
full search algorithm.
40
PERFORMANCE COMPARISON OF SAD AND WSAD BASED ON
FULL SEARCH ALGORITHM IN BIT-RATE (W1 10W2 7)
41
Performance Comparisons of Various Search
Algorithms
42
Comparisons
43
Comparisons