Introduction to MPEG7

1
Introduction to MPEG-7
Young Gyun, Kim
Kumoh National University of Technology
ygkim_at_cespc1.kumoh.ac.kr
2
Summary of MPEG
The Moving Picture Experts Group (MPEG) is a
working group of ISO/IEC in charge of
the development of international standards for
compression, decompression, processing, and
coded representation of moving pictures, audio
and their combination. So far MPEG has produced
MPEG-1, the standard for storage and
retrieval of moving pictures and audio on
storage media (approved Nov. 1992)
MPEG-2, the standard for digital television
(approved Nov. 1994) MPEG-4 version 1, the
standard for multimedia applications (approved
Oct. 1998) and is now developing
MPEG-4 version 2 (to be approved December 1999)
MPEG-7 the content representation standard for
multimedia information search, filtering,
management and processing (to be approved July
2001).
3
1. Context
Many text-based search engine are available
Identifying Information
Not possible for audio-visual content
Not possible to efficiently search the web for
Multimedia contents
King Lear congratulates his assistants on the
night after the battle,
Multimedia databases
Colour,texture,shape of objects in the picture
4
2. MPEG-7 Objectives(1)
Multimedia Content Description Interface
Specifiy a standard set of descriptors that can
be used to describe various types of
multimedia information
Standardise ways to define other descriptors as
well as structures(Description Schemes) for the
descriptors and their relationships
Description Definition Language(DDL) Standardise
a language to specify description schemes
5
2. MPEG-7 Objectives(2)
Indexing, Searching AV material that has MPEG-7
data associated with it
Still Pictures, Graphics, 3D Models, Audio,
Speech,Video, and combined information
MPEG-7 descriptors
Not depend on the ways the described content is
coded or stored
Not depend on the (coded) representation of the
material
6
2. MPEG-7 Objectives(3)
Descriptive features
The same material can be described using
different types of features, tuned to the area
of application
Visual material description of shape,size,
texture, colour,movement and position
Audio material description of key,tempo, tempo
changes, position in sound space
The level of abstraction
Many low-level features can be extracted
in fully automatic ways High level features
need(much) more human interaction
7
2.1 What is the Scope of the Standard?(1)
Feature extraction Neither automatic nor
semi-automatic feature extraction algorithms
will be inside the scope of the standard
Search engines will not be specified within the
scope of MPEG-7
Feature extraction Standard description
Search engine
Scope of MPEG-7
Figure 1. Scope of MPEG-7
8
MPEG-7 will standardise
Multimedia Content Description Interface
A set of description schemes and descriptors
A language to specify description schemes, I.e. a
Description Definition Language(DDL).
A scheme for coding the description
9
2.1 What is the Scope of the Standard?(2)
Descriptor, Description Scheme, and DDL
DDL
DS
DS
D
D
D
DS
D
D
Not in standard defined using DDL
Defined in Standard
Figure 2. An abstract representation of possible
relations between Ds and DSs
10
2.1 What is the Scope of the Standard?(3)
The DDL provides the mechanism to built a
description scheme which forms the basis for the
generation of a description
MPEG-7 Description Definition Language(DDL)
Description Generation
MPEG-7 Description Schemes Descriptors
Figure 3 The role of Ds and DSs for the
generation of descriptions
11
2.1 What is the Scope of the Standard?(4)
The emphasis of MPEG-7 will be the provision of
novel solutions for audio-visual content
description
Search/Query Engine
D,DS
Description Generation
User
MPEG-7 Descriptor
Coded MPEG-7 Description
Encoder
Decoder
Filter Agents
MM Content
Figure 4 An Abstract representation of possible
applications using MPEG-7
12
3. Areas of Interest(1)
Applications
Digital Libraries(Image Catalogue, Musical
Dictionary)
Multimedia directory services(e.g. yellow pages)
Broadcast media selection(radio channel, TV
channel)
Multimedia editing(news service, media authoring)
Application Domains
Journalism
Education
Shopping
Bio-medical applications
Tourist Information
Entertainment
Remote sensing
Film, Video and Radio archives
13
3. Areas of Interest(2)
Music
Play a few notes on a keyboard and get in return
a list of musical pieces or close to the
required tune
Graphics
Draw a few lines on a screen and get in return a
set of images containing similar graphics,
logos, ideograms,...
Image
Define objects, including colour patches or
textures and get in return example images
14
3. Areas of Interest(2)
Voice
Using an excerpt of Pavarottis voice , and
getting a list of Pavarotti records, video clips
where Pavarotti is singing or video clips where
Pavarotti is present
15
MPEG-7 based data search
Initial Descriptor Table


1011 text link 1011 text
link 1011 text
link
Feature selection
Search engine
Interface
Interface
MPEG-7 decoder
MPEG-7 stream
MPEG-7 database

AV content

Query result (link address)
Presentation Engine
Interface
Interface
AV content
AV data decoder
AV content
AV stream
Client
Server
Example of a client-server architecture in a
MPEG-7 based data search
16
4. Method of Work and Work Plan
The method of development
is comparable to that of the previous MPEG
standards
The Work Plan for MPEG-7
Call for Proposals October
1998 Working Draft
December 1999 Committee Draft
October 2000 Final Committee Draft
February 2001 Draft International
Standard July 2001 International
Standard September 2001
17
5. FAQ(1)
What is MPEG-7 ?
A Standardised description of various types of
multimedia information
Why is MPEG-7 needed?
More and more audio-visual information is
available. At the same time, the increasing
availability of potentially interesting material
makes this search more difficult.
18
5. FAQ(2)
Who is currently participating in the
development of the MPEG-7 standard ?
Broadcasters, equipment manufacturers, digital
content creators and managers, transmission provid
ers, publishers, university researchers
Will MPEG-7 include audio or video content
recongnition? The standardisation of
audio-visual content recognition tools is beyond
the scope of MPEG-7. MPEG-7 will concentrate on
standardising a representation that can be used
for description.
19
5. FAQ(3)
Will the standard allow automatic extraction of
descriptions as well as manual entry? The
descriptions that conform to the MPEG-7 standard
could be entered by hand, but they could also be
automatically extracted. Some features can be
best extracted automatically(colour, texture)
When will MPEG-7 replace the existing MPEG-1 and
MPEG-2 standards?
MPEG-7 will not replace MPEG-1, MPEG-2 or in fact
MPEG-4 it is intended to provide complementary
functionality to these other MPEG standards
20
5. FAQ(4)
What is the relationship between MPEG-7 and
other MPEG activities? MPEG-7 can be used
independently of the other MPEG standards. The
description might even be attached to an analog
movie. The representation that is defined within
MPEG4. In addition, MPEG-7 descriptions could
be used to improve the functionality of previous
MPEG standards.
21
6. Conclusion
A Content-Based Retrieval is coming soon
A Content-Based Search Engine
Visual Information Retrieval(VIR)
A Knowledge-Based Retrieval
A Knowledge Representation
AI-Based Retrieval
22
7.References
1 MPEG Requirements Group, MPEG-7 Requirements
Document, Doc. ISO/MPEG N2461, MPEG
Atlantic City Meeting, October 1998 2 MPEG
Requirements Group, Applications for MPEG-7,
Doc. ISO/MPEG N2462, MPEG Atlantic City
Meeting, October 1998 3 MPEG Requirements
Group, MPEG-7 Evaluation Procedure, Doc.
ISO/MPEG N2463, MPEG Atlantic City Meeting,
October 1998 4 MPEG Requirements Group,
MPEG-7 Proposal Package Description(PPD),
Doc. ISO/MPEG N2464, MPEG Atlantic City Meeting,
October 1998