Multimedia Lab

1

• Multimedia Lab
• Rik Van de Walle
• Department of Electronics and Information Systems
• Ghent University - IBBT
• Gaston Crommenlaan 201 bus 8
• B-9050 Ledeberg-Ghent, Belgium
• t 32 9 33 14914
• f 32 9 33 14896
• m 32 478 39 62 31
• t secr 32 9 33 14911
• e rik.vandewalle_at_ugent.be
• URL multimedialab.elis.ugent.be

2
Multimedia Lab - UGent

• History
• background in image and signal processing
• 1999 Multimedia Systems and Applications
• new research area within Ghent University
• 2001 Multimedia Lab formally founded
• new research group within ELIS department
• 2003 link/co-operation with IMEC
• 2004 partner of the IBBT
• www.ibbt.be, Flemish Government

3
Multimedia Lab - UGent

• People
• 5 staff members
• Rik Van de Walle, full professor
• Peter De Neve, 10 lecturer
• Erik Mannens, project management
• Peter Lambert, senior researcher
• Ellen Lammens, administrative management
• currently 25 researchers
• PhD students
• others (projects)
• about 15 master students per year

4
Multimedia Lab - UGent

• Teaching
• programs
• ba/ma Computer Science (Faculty of Engineering)
• ba/ma Electronics (Faculty of Engineering)
• ba/ma Informatics (Faculty of Sciences)
• Multimedia Techniques (baC3 - baE3)
• Development of MM Applications (maC1 - maE1)
• Advanced MM Applications - project (maC2 - maE2)
• Multimedia (baINFO2)
• Internet Technology (baINFO3)

5
Research topics data metadata
opera il barbiere di Siviglia composer
Gioachino Rossini director Claudio Abbado type
audio format MP3
opera il barbiere di Siviglia composer
Gioachino Rossini libretto Cesare Sterbini type
web page format html
place Sydney building opera house architect
Jørn Utzon type still image format JPEG2000
6
Research topics adaptation - optimization
7
Research topics (1)

• Advanced video applications
• development and exploitation
• of scalable video compression techniques
• motion estimation and compensation
• ROI-based video compression (Region of Interest)
• adaptation of multimedia data
• with respect to varying usage environments
• development of iDTV applications
• (a/o based on MHP, the Multimedia Home Platform)

8
Research topics (2)

• Mobile multimedia applications
• reduction impact of network losses
• on quality of multimedia presentations
• (a/o adaptive coding/decoding
• wrt varying network characteristics)
• rendering multimedia data
• on mobile terminals with limited resources
• transparent handover of multimedia sessions
  between different devices (session mobility)
• hardware/software co-design
• of embedded multimedia systems

9
Research topics (3)

• Standardization
• MPEG-21
• Digital Item Declaration
• Digital Item Adaptation
• Digital Item Processing
• Development of Reference Software for various
  tools
• MPEG-4 Scalable Video Coding (SVC)
• MPEG-4 Advanced Video Coding (H.264/AVC)
• JVT - Joint Video Team
• VQEG - Video Quality Experts Group
• IFTA - International Federation of Television
  Archives

10
Ex. 1 exploitation of scalability
11
Ex. 2 session mobility for mobile applications
12
Ex. 3 video adaptation for mobile applications
13
In short transparent content adaptation
14
New Research Pathsrelated to MMSEM
15
Feature Extraction (Video)

• Use cases
• Video surveillance
• Detection of moving foreground objects
• Video adaptation
• Change bitrate, framerate, resolution, coding
  format,
• goal reduction of network load, processing
  load, adjustment to client capacities or user
  preferences
• Extraction of relevant features ROI, semantics,
  shot detection

16
Feature Extraction tool

• Low level extraction

17
Integrated Video Surveillance System

• Combining expertise

MPEG-21 Digital Item
Raw Pixel Data
H.264/AVC
features
features
18
Interoperable Integrated Video Surveillance System

• Issues
• Which features (high level lt-gt low level,
  automatic lt-gt manual)
• How to describe them
• When to describe them (camera, video processing
  module, network node, client)
• How to link them to video resource (storage,
  distribution, presentation, )

New Use Case?
19
New IBBT Projectsrelated to MMSEM
20
PeCMan

• The goal of the PeCMan project is to research the
  major technological challenges of a user-centric
  solution enabling uniform storage and
  manipulation of data as well as universal access
  to this personal data. A successful solution will
  have premium capabilities in the areas of
  security, performance, usability and description
  of the personal data

21
Personal Digital Photo Archives --gt need for
effective, uniform annotation strategy
semantic gap is still very wide!

• Overcome semantic gap?
• Extract the good features -gt which algorithm?,
  domain specific?
• Combine which primitive features for scene
  understanding?
• Machine learning -gt many methods, training on
  different datasets, ...
• Ontologies -gt need to model the whole world?
  Content vs. Context

22
Feature extraction - human vision system best
tuned to color and texture perception - imitate
cells of the visual cortex - texture features can
provide semantic information - texture helpful
in object segmentation
23

• Machine Learning
• - supervised lt-gt unsupervised
• - self-organizing map processing principles of
  human (visual auditory) cortex
• incremental learning knowledge must be
  continuously updated to manage
• non-static archives, relevance feedback

Feature extraction
Ontologies
Classification
Machine learning
Scene understanding
--gt use both manual and automatic annotation with
minimal user overhead
24
Lindo
25
FIPA / AVEQ

• investigate the processes related to engineering
  and analysis of audiovisual material, based on
  the notion of a virtual model of the product
• explore some aspects of reverse engineering,
  including audiovisual-quantization, analysis and
  classification
• Stretching the idea of the virtual model, we may
  develop a much more powerful analysis and
  classification system when we succeed in
  reconstructing the model the background, the
  characters, the objects that constitute the scene
  of the image quantifying the subjects per scene
  by a finite list of normalized and relevant
  vectors and being able to express semantics,
  history and behaviour in a single geospatial and
  temporal reference

26
Industrial partners

• Main partners (projects MPEG)
• Alcatel, France-Belgium
• Artec/Televic, Belgium
• Barco, Belgium
• Belgacom, Belgium
• CISCO/Scientific Atlanta, USA
• Concentra, Belgium
• Deutsche Telecom, Germany
• eTampere, Finland
• France Telecom, France
• IBM, USA
• Intel, USA
• LG Electronics, Korea

27
Industrial partners

• Main partners (projects MPEG)
• Microsoft Research, USA
• Museon, Netherlands
• Philips, Netherlands
• Porthus, Belgium
• ScreenPeaks, Israel
• Siemens, Germany
• Sun Microsystems, USA
• Telindus, Belgium
• Thomson Multimedia, France
• T-Systems Nova GmbH, Germany
• VMMa, Belgium
• VRT, Belgium

28
Contact

• Erik Mannens
• Department of Electronics and Information Systems
• Ghent University - IBBT
• Gaston Crommenlaan 201 bus 8
• B-9050 Ledeberg-Ghent, Belgium
• t 32 9 33 14992
• f 32 9 33 14896
• m 32 478 39 62 31
• t secr 32 9 33 14911
• e erik.mannens_at_ugent.be
• URL multimedialab.elis.ugent.be