Multimedia Information Systems

1
Multimedia Information Systems

• Multimedia Information Systems an Introduction

2
Contents

• Definition
• Multimedia Humans and Computer
• Why Multimedia?
• Information Flow in Multimedia
• Some Key Factors
• Enabling Factors
• Media Types

3
Definition Multimedia

• Media plural for medium, a mode of transmission
  and exchange.
• Multimedia multiple modes of transmission and
  exchange
• Multimedia could hence refer to a collection of
  many streams like audio, video, images, text,
  haptic etc.
• Clear distinction between electronic media and
  natural media.

4
Information

• Data vs information?
• Extraction of information is a challenging task
• Information processing is an active research area
  in statistics, computer science, mathematics.

5
System

• System refers to a working entity a collection
  of modules physical, logical or virtual working
  together to solve a problem
• Systems engineering refers to designing,
  configuring, and maintaining systems

6
Multimedia Information Systems

• Multimedia Information Systems refers to a branch
  of study wherein systems are designed to extract,
  create, manage, process and present information
  from multimedia data.

7
Why Multimedia?

• Engages Senses
• Increases attention, understanding and retention.
• A more intuitive way of conveying information.

8
Why Multimedia-II?

• Images
• classification
• Retrieval
• Enhancement
• Content Based Analysis
• Text
• conceptual modeling
• syntax based vs. semantics based retrieval
• Integrate text with charts, photographs, etc.
• Sound
• speech synthesis and digitized voice
• Content Based Analysis

9
Multimedia Humans

• Humans use five senses to communicate and
  interact with their environments
• Biological, neurological and psychological
  principles guide how humans sense, perceive and
  cognize information.
• Many of these processes are not well-understood
  enough to be mimicked.

10
Multimedia Computing

• Computer interact with humans through mouse,
  monitor, keyboard etc.
• Sensory capabilities in a computer are not even
  close in sophistication, design and configuration
  of the human sensory system
• Computers hence have no sensory capability

11
Multimedia The unfortunate paradox

• Our final customer is the human sensory system.
• Traditional computer input output system are not
  natural.
• Hence the paradox is to adapt computing hardware
  software for the final customer

THIS IS NOT EASY!
12
So how do we do it?
Generalized Multimedia Information Flow
13
Some Key Factors
14
Key Factors Design

• Computers have no sensory capabilities
• Interactive
• Video without interaction is TV
• Multiple input-output (media) Streams involved
• Interfaces assume very high priority
• Use knowledge from human sciences

15
Key Factors Implementation

• Voluminous Data Handling Capability
• Inter-operability between standards
• Co-occurrence analysis between events in various
  media.
• Sophisticated synchronization models
• Different and more elaborate software engineering
  principles

16
MlS Multi-disciplinary Approach

• Digital signal processing
• Image processing
• Computer graphics
• Digital Video technology
• Visual interactive languages
• Advanced databases
• Operating systems
• Computer networks
• Psychology, Cognitive Science
• Arts

17
Multimedia Revolution

• Enabling Factors

18
Enabling Factors-I

• Until recently, multimedia meant text and images
• Document image management systems are outgrowth
  of the facsimile technology
• As the document image systems became more
  data-intensive, compression standards were
  developed

19
Enabling Factors-II

• Optical disc technology
• large volumes
• preserve quality
• high accuracy
• Shift from analog to digital processing
• high resolution image
• high fidelity audio
• high definition video
• nicely typeset text.

20
Enabling Factors-III

• Fiber optics technology
• high bandwidth
• accuracy
• speed
• Fast specialized processors
• audio boards
• video boards
• Better software,
• real-time OS
• codec (software only)
• Object oriented paradigm and OO modeling

21
Some Media Types

• A basic Introduction

22
Media Types

• Image
• Video
• Sound
• Text
• Data
• Haptics

23
Images and Videos

• What is an image?
• To the human visual system
• computationally
• Analog
• Digital.
• What is a video?
• Video is a temporal sequence of images.

24
Image Analog Modeling
A band of Infinitely small width
25
Image Digital Modeling
A band of some small width
26
Challenges for Image Analysis

• Image Representation in a computer understandable
  form
• Image modeling
• Image Capture
• Image compression
• Image content representation
• Computational image analysis
• Image enhancement and processing
• Image content analysis

27
Image Modeling

• Digital Images are modeled as a matrix
• Every element of the matrix is called a pixel
• A value is given at every pixel gives the images
• Binary value
• Grayscale value at that place
• RGB values

28
Pixel

• Pixel Picture Element
• Outgrowth of the digital system.
• Requires quantization and digitization of the
  analog signal.

29
Image Capture

• Analog
• Conventional camera
• Digital
• Digital cameras
• Scanners

30
Types of Images

• Face photos Mug shots, casting
• Finger prints
• CAT-scan data
• 3D images.
• Geographical Images
• News photos (general. Usually accompanied with
  explanatory text)
• Surveillance (semi-specific domain, we may or may
  not know what to look for).
• Video segments
• Movies general purpose
• surveillance
• News broadcast
• Sports

31
Image Compression

• The number of data points are very large
• Large data points means chance of data
  redundancy a basic factor in every compression
  scheme.
• Compression is inversely related to image
  quality.
• There are many compression schemes with each
  having advantages and disadvantages.

32
Image Content Representation

• So how can pixels let me know the image contains
  a tree?
• Image meta-data
• This is one of the biggest challenge wanna do a
  phd?

33
Computational image analysis
34
Image Processing

• Image enhancement
• Image Restoration
• Edge detection
• Image segmentation
• Image Morphology
• Erosion
• dilation

35
Image Content Analysis

• Use clues from image processing to develop tools
  for image content analysis
• Bottom up vs top down approach
• Where does the content lie?
• Image content
• Visual content
• Semantic content
• Affective content

36
Sound and Audio

• Audio as opposed to Images is a one dimensional
  signal
• 3D audio refers to placement of audio signals in
  3D space and their analysis
• Humans can hear signals between the frequency
  range of 20hz to 20,000 hz.
• Music is a form of audio with special needs

37
Audio Application areas

• Many application areas movies, recording
  industry
• Telephone answering, voice mail
• Voice encryption
• Cellular radio technology
• Transmission of voice, image and data over network

38
Computational Audio

• Audio Capture
• Analog
• Digital
• Analog to Digital
• Audio Modeling, Compression
• Audio Enhancements and Processing
• Audio Content Analysis and Retrieval

39
Digital Speech Processing

• Speech coding
• voice annotations
• videoconferencing
• voice mail
• Speech recognition
• dictation
• voice data entry
• voice activated response system
• Speech synthesis
• information access (remote)
• text proofreading
• conversion of e-mail to audio

40
Text Characteristics

• Symbolic
• Structured/Grammar Rules
• Non- redundant Any change in text will cause a
  change in its semantics/meaning.
• Easy to store Requires little storage space.
• Easy to transmit Requires little bandwidth
• sensitive to loss during transmission

41
Computational Text

• Modeling of Text
• ASCII
• 7 Bit digital Display
• DPI
• Text Acquisition
• Text Entry
• Character Recognition
• Hand Writing Recognition
• Text Processing
• Content Analysis
• Natural Language Processing
• Xml, sgml,html

42
Haptics

• Sensation of Touch and modeling of the haptic
  experience
• Recent Addition to the multimedia experience
• Video games with force feedback a common place
  now
• Intelligent Processing of tactile data is still
  an active research area.

43
The Next Class

• Fundamentals of Multimedia and Signal Processing.

44
References

• Digital Image Processing Gonzales and Woods.
• Web sources
• Multimedia Computing, Communication Applications
  by Steinmetz and Nahrstedts