MPEG4 Standard for Content Creation

1
Université Catholique de Louvain Faculté des
Sciences Appliquées
Laboratoire de Télécommunications et
Télédétection (TELE)

• MPEG-4 Standard for Content Creation
• and Multimedia Applications of the Future

François Xavier Fanard Monica Gemo Jean-Julien
Filatriau
2
Moving Picture Expert Group Formats

• MPEG-1 STANDARD FOR STORAGE AND
  RETRIEVEL OF AV MATERIAL
  ON STORAGE MEDIA
• MPEG-2 STANDARD FOR DIGITAL TV
• Pay TV
• DVD
• HD TV
• MPEG-4 CONTENT-BASED CODING OF
  MULTIMEDIA PRESENTATION

3
Motivation

• COMMON TECHNOLOGY for multimedia
• ALLOWING MORE AND VERSATILE INTERACTIVITY
• INTEGRATING NATURAL AND SYNTHETIC CONTENT
  (augmented reality application)
• COVERING WIDE RANGE OF ACCESS CONDITIONS (PDAs,
  Cellphones, Digital TV, Internet, etc.)
• INTELLECTUAL PROPERTY MANAGEMENT AND PROTECTION
  (IPMP)

4
Traditional Paradigm for Production, Delivery,
Consumption
Audio/Video Objects Natural/Synthetic 2D/3D
Rigid Format !
A/V
5
MPEG-4 Paradigm for Production, Delivery,
Consumption
FLEXIBLE PRESENTATION
6
Example MPEG-4 Scene
7
Scene Description
Hierarchical Structure

• Each scene Collection of Objects instead of
  Pixels/Samples
• AVO video and/or audio objects
• AVO can group more objects

Scene Graph
Father and child (arbitrarily shaped object)
Web site (application screen)
HDTV set
Furniture (synthetic)
Floor and wall (sprites)
TV frame (synthetic)
HomeDVD
Crib
Lamp
Side-table
Audio object
Audio
Video
8
Encoding/DecodingProcess
Interactive audiovisual scene
Delivery layer
9
BIFS Binary Format for Scenes

• Language for describing and dynamically change a
  scene
• VRML concepts set of nodes to represent the
  primitive scene objects to be composed, the
  behavior and interactivity
• Encode the scene in binary formatSupport
  Streaming
• Modify spatio-temporal layout Add/delete objects
  and nodes
• Change visual/acoustic properties of OBJECTS
• Colour, position, texture
• Compression efficiency

10
Interactivity with Media Objects

• Author may allow the user to
• Change the viewing and listening position of the
  scene
• Place media objecs anywhere in a given coordinate
  system
• Apply streamed data to media objects
• Select language when multiple languages are
  available

11
Elementary Streams
Interactive Scene Description
ES for Scene Description
ES for Object Descriptors
Object Descriptors

• associated with AVOs
• enable manipulation of the AVO

Visual Stream
Visual Stream
Visual Stream
Audio Stream
12
Video Coding
13
VOP Extraction
14
Sprites code unchanging backgrounds
15
Applications

• Real time communication
• Telepresence, Augmented Reality
• Content based storage and retrieval
• Broadcast
• Digital TV

16
Examples Dynamic/Interactive Content
http//www.research.ibm.com/mpeg4/Demos/DemoSystem
s.htm
Transitions with multiple simultaneous videos
TicTacToe MPEG4 manages the overall rendering