Multimedia Databases

1
Multimedia Databases

• Assaf Waizman Tal Haviv

2
In this presentation

• Introduction.
• What is a Multimedia Database?
• Why do we need it?
• Modeling Multimedia Databases.
• Core-ER diagrams.
• SQL extension.
• Multimedia database searching.
• Exact match Vs. similarity.
• Manual Indexing Techniques.
• Automatic Indexing Techniques

3
What is a Multimedia Database?

• A collection of multimedia data items.
• images, graphic objects, video, audio, etc.
• These data types differ in characteristics and
  hence require different methods for optimal
  organization and management like storage, access,
  indexing and retrieval.

4
Why Do We Need It?

• With everyday advances in communication,
  computing and storage technologies, multimedia
  applications are on a high.
• Most applications use multimedia data.
• Querying for digitized information.
• A picture is worth a thousand words.
• And much more...

5
Some examples

• Training and education.
• Online lectures, presentations etc.
• Entertainment industry.
• Online Movies, clips etc.
• Archival of images captured by satellites.
• Medical databases containing X-rays and MRIs.

6
Some examples

• Front end of Yahoo Photos Multimedia Database.

7
The problem

• Data means not just numbers and small strings but
  multimedia data as well.
• Databases promise
• Well structured data organization.
• Efficient storage of large amounts of data.
• Querying.
• Transactional support for concurrent users.

8
The problem

• If you include multimedia data, you want
• All of the above.
• Tight integration between standard data and
  multimedia data.
• And yet
• Multimedia data is large and may swamp other data
  (not in 1NF).
• multimedia data structures are completely
  different from standard database structures.
• multimedia data structures do not easily lend
  themselves to content-based searching.

9
Data integration

• Databases already integrate various kinds of
  data, numbers, dates, small text strings.
• They do this by the use of domains, each one has
  2 aspects
• A range of values which are acceptable.
• Some operations (date handling etc.).

10
Domain Types for Multimedia Data

• DBMS typically provide three different kinds of
  domain for multimedia data
• large object domains- BLOBs CLOBs (unstructured
  sequence of bytes/ chars).
• file references- instead of holding data, hold a
  pointer.
• genuine multimedia data types.
• large objects at best allow you to extract
  sections or to concatenate them.
• file references- all data at once or none.

11
Searching Multimedia Data

• A DBMS permits a user to search the database by
  content.
• give the name of the student with ID 0123456.
• We would like to do the same with multimedia
  data.
• give the pictures painted by Da Vinci.
• give me some pictures with black in the lower
  left corner.
• give me some sounds files of loud and unpleasant
  Opera.

12
Modeling multimedia databases

• What information should be held in the database.
• The types of relationships held between the data
  items.
• Developers and users must understand how the data
  is modeled in order to sufficiently develop,
  maintain and query the database.

13
Modeling Multimedia Databases

• The modelling of multimedia databases is a far
  more complex task than that of conventional
  databases.
• Dynamic behavior of multimedia objects- its Hard
  to represent a movie only by some attributes.
• Advanced queries that can be run on the data.

14
Modeling Multimedia Databases

• Standard ER diagrams.

Entity
Attribute
15
Modeling Multimedia Databases

• Core-ER extends the basic ER diagram to
  incorporate core objects- real world objects such
  as sound, graphic and video files

Core entity
Attributes
16
Modeling Multimedia Databases
Relationship between entities
17
Modeling Multimedia Databases

• A dashed rectangular box surrounds any attribute
  of an entity which can be considered core.

The picture physically held within the database
18
Modeling Multimedia Databases

• Methods, or valid queries, for the database are
  shown in double-edged diamonds.

Inputs
Output
19
SQL extension

• Following the creation of an ER-Core diagram we
  need an extension to SQL to allow us to create
  the database. The extension allows for a special
  attribute core which is used to store the
  object itself. This means that the core object
  can be distinguished from the attributes, which
  describe it.

20
SQL extension

• Create table MOVIE
• (TITLE VARCHAR (30),
• DIRECTOR VARCHAR (30),
• YEAR INTEGER,
• STUDIO VARCHAR (30),
• CORE))
• Declare External AppearsIn (Movie, Actor) Returns
  BOOLEAN
• ltFunction Definitiongt

21
Multimedia Database Searching

• When querying in a standard relational database
  the user can expect a perfect data match for the
  requested query.
• Give me the names of all criminals who have been
  convicted of house break-ins over the last six
  months.

22
Multimedia Database Searching

• However this is not the case with multimedia
  databases.
• The user has a photo-fit
• of a single suspect and
• is required to search
• the database for a match.
• An exact match can only be retrieved when any of
  the stored objects are actually bit for bit
  identical to the search image.

23
Multimedia Database Searching

• Thus when querying this type of data approximate
  matching techniques are used and the system
  measures the degree of similarity between these
  approximate matches.

Image
Similar
Yes/No
24
Multimedia Database Searching

• Similar images

25
Multimedia Database Searching

• Q Why cant we store a textual description of
  the multimedia object, describing what they
  contain? Wouldnt that give us a close match?
  (Manual Indexing Techniques).
• A There are almost infinitely different ways to
  describe a picture, or a song, or a film,
  depending on a users point of view.

26
Multimedia Database Searching

• The image could be described differently by
  different people. One person may describe the
  picture as city while another person may state
  that it is river another still would say it is
  landscape.

27
Multimedia Database Searching

• Automatic Indexing Techniques
• The query processor builds a color histogram of
  the sample image on which to query on.
• The histogram is now compared against the
  histogram of the other images held within the
  database for their similarity.
• The matches are then ranked and placed in order
  of their ranking for the user.

28
Multimedia Database Searching
Query
Color histogram

Rank 6 Similarity 0.66
Rank 7 Similarity 0.62
Rank 1 Similarity 0.87
29
ThThThats All Folks!