Anandi Giridharan

1
MultimediaStorage Techniques

• Anandi Giridharan
• Electrical Communication Engineering,
• Indian Institute of Science,
• Bangalore 560012, India

2
Media and Storage Requirements

• Audio, video and image require vast amount of
  data for their representation.
• There are 3 main reasons for compression
• Large storage
• Doesn't allow playing back uncompressed
  multimedia data.
• Network bandwidth
• Storage requirements for multimedia application
  such as encyclopedia is
• 500,000 pages of text (2 KB per page) total 1 GB
• 3000 color picture total 3 GB.

3
Text 8x8 pixel / char Vector Graphics A
typical still image is composed of 500 lines.
Each line is defined by horizontal and vertical
position and an 8 bit attribute field. Horizontal
axis is represented by 10 bits log2(640) and
vertical axis by 9 bits log2(480). Uncompressed
audio If a sampling rate of 8kHz is used, and
data is quantized at 8 bits/sample then 64
kbits/sec is required. CD quality stereo audio
Sampling rate used is 44.1 kHz, hence at 16
bits/sample 44.1 x 16 705.6 kbits/sec is
required. Interaction between human users via
multimedia information is involved for example,
requires that the end to end delay should not
exceed 150 ms. Hence compression method should be
selected
How much storage is required for different data
types
4
DIGITAL VIDEO

• Digital video is just a digital representation of
  the analogue video signal.
• Unlike analogue video that degrades in quality
  from one generation to the next, digital video
  does not. Each generation of digital video is
  identical to the parent.
• With digital video, four factors have to be kept
  in mind.
• Frame rate
• Spatial Resolution
• Colour Resolution
• Image Quality

5
Frame Rate

• The standard for displaying any type of non-film
  video is 30 frames per second (film is 24 frames
  per second). Additionally these frames are split
  in half (odd lines and even lines), to form what
  are called fields.
• When a television set displays its analogue
  video signal, it displays the odd lines (the odd
  field) first. Then is displays the even lines
  (the even field).
• Each pair forms a frame
• and there are 60 of these fields displayed every
  second (or 30 frames per second). This is
  referred to as interlaced video.

6
After processing the fragment on the left by the
FRC filter the frame rate increased 4 times
Fragment of the "matrix" sequence (2 frames)
7

• A computer monitor, however, uses a process
  called "progressive scan" to update the screen.
• With this method, the screen is not broken into
  fields. Instead, the computer displays each line
  in sequence, from top to bottom.
• This entire frame is displayed 30 times every
  second. This is often called non-interlaced video.

8

• Colour Resolution
• This second factor is a bit more complex. Colour
  resolution refers to the number of colours
  displayed on the screen at one time. Computers
  deal with colour in an RGB (red-green-blue)
  format, while video uses a variety of formats.
  One of the most common video formats is called
  YUV.

9
This test table was used to estimate the color
resolution. First we determine the border when
one of the colors on the resolution chart
disappears, and color sharpness is found on the
scale on the right.
10
Spatial Resolution

• The third factor is spatial resolution - or in
  other words, "How big is the picture?". Since PC
  and Macintosh computers generally have
  resolutions in excess of 640 by 480,
• The National Television Standards Committee (
  NTSC) standard used in North America and Japanese
  Television uses a 768 by 484 display.
• The Phase Alternative system (PAL) standard for
  European television is slightly larger at 768 by
  576.

11
Spatial resolution is a parameter that shows how
many pixels are used to represent a real object
in digital form. Fig. 2 shows the same color
image represented by different spatial
resolution. Left flower have a much better
resolution that right one
12
Image quality

• The final objective is video that looks
  acceptable for your application.
• For some this may be 1/4 screen, 15 frames per
  second (fps), at 8 bits per pixel.
• Other require a full screen (768 by 484), full
  frame rate video, at 24 bits per pixel (16.7
  million colours).

13
(No Transcript)
14
Need For Compression

• How the four factors mentioned above (frame rate,
  colour resolution, spatial resolution and image
  quality) affect your selection.
• With more colours, higher resolution, faster
  frame rates and better quality, you will need
  more computer power and will require more storage
  space for your video.
• 24-bit colour video, with 640 by 480 resolution,
  at 30 fps, requires an astonishing 26 megabytes
  of data per second! Not only does this surpass
  the capabilities of the many home computer
  systems, but also overburdens existing storage
  systems.

15

• 640 horizontal resolution
• X 480 vertical resolution
• 307, 200 total pixels per frame
• X 3 bytes per pixel
• 921, 600 total bytes per frame
• X 30 frames per second
• 27, 648, 000 total bytes per second
• / 1, 048 576 to convert to megabytes
• 26.36 megabytes per second!
• Calculation to show space required for video is
  excessive

16
Factors Associated with Compression

• The goal of video compression is to massively
  reduce the amount of data required to store the
  digital video file, while retaining the quality
  of the original video
• Real-Time versus Non-Real-Time
• Symmetrical versus Asymmetrical
• Compression Ratios
• Lossless versus Lossy
• Interframe versus Intraframe
• Bit Rate Control

17
Real-Time versus Non-Real-Time

• Some compression systems capture, compress to
  disk, decompress and play back video (30 frames
  per second) all in real time there are no
  delays.
• Other systems are only capable of capturing some
  of the 30 frames per second and/or are only
  capable of playing back some of the frames.
• Insufficient frame rate is one of the most
  noticeable video deficiencies.
• Without a minimum of 24 frames per second, the
  video will be noticeably jerky. In addition, the
  missing frames will contain extremely important
  lip synchronisation data.
• If the movement of a person's lips is missing due
  to dropped frames during capture or playback, it
  is impossible to match the audio correctly with
  the video.

18
Real time Non Real time
19
Symmetrical Versus Asymmetrical

• This refers to how video images are compressed
  and decompressed. Symmetrical compression means
  that if you can play back a sequence of 640 by
  480 video at 30 frames per second, then you can
  also capture, compress and store it at that rate.
  
• Asymmetrical compression means just the opposite.
  The degree of asymmetry is usually expressed as a
  ratio. A ratio of 1501 means it takes
  approximately 150 minutes to compress one minute
  of video.
• Asymmetrical compression can sometimes be more
  elaborate and more efficient for quality and
  speed at playback because it uses so much more
  time to compress the video.
• The two big drawbacks to asymmetrical compression
  are that it takes a lot longer, and often you
  must send the source material out to a dedicated
  compression company for encoding

20
Compression Ratio

• The compression ratio relates the numerical
  representation of the original video in
  comparison to the compressed video.
• For example, 2001 compression ratio means that
  the original video is represented by the number
  200. In comparison, the compressed video is
  represented by the smaller number, in this case,
  that is 1.
• With MPEG, compression ratios of 1001 are
  common, with good image quality.
• Motion JPEG provides ratios ranging from 151 to
  801, although 201 is about the maximum for
  maintaining a good quality image.

21
Lossless Versus Lossy

• The loss factor determines whether there is a
  loss of quality between the original image and
  the image after it has been compressed and played
  back (decompressed). The more compression, the
  more likely that quality will be affected.
• Virtually all compression methods lose some
  quality when you compress the data the only
  lossless algorithms are for still image
  compression.
• Lossless compression can usually only compress a
  photo-realistic image by a factor of 21.

22
Interframe Versus Intraframe
One of the most powerful techniques for
compressing video is interframe compression.
Interframe compression uses one or more earlier
or later frames in a sequence to compress the
current frame, while intraframe compression uses
only the current frame, which is effectively
image compression.
Since interframe compression copies data from one
frame to another, if the original frame is simply
cut out (or lost in transmission), the following
frames cannot be reconstructed properly.
Making 'cuts' in intraframe-compressed video is
almost as easy as editing uncompressed video
one finds the beginning and ending of each frame,
and simply copies bit-for-bit each frame that one
wants to keep, and discards the frames one
doesn't want. Another difference between
intraframe and interframe compression is that
with intraframe systems, each frame uses a
similar amount of data.
23
Bit Rate Control

• A good compression system should allow the user
  to instruct the compression hardware and software
  which parameters are most important.
• In some applications, frame rate may be of
  paramount importance, while frame size is not.
• In other applications, you may not care if the
  frame rate drops below 15 frames per second, but
  the quality of those frames must be of very good.

24
MPEG (Moving Picture Expert Group)
MPEG was set standard for Audio and Video
compression and transmission MPEG-1 is a standard
for lossy compression of video and audio. It is
designed to compress VHS-quality raw digital
video and CD audio down to 1.5 Mbit/s (261 and
61 compression ratios respectively) without
excessive quality loss, making Video CDs, digital
cable/satellite TV and digital audio broadcasting
(DAB) possible. MPEG-1 has become the most widely
compatible lossy audio/video format in the world,
and is used in a large number of products and
technologies. The best-known part of the MPEG-1
standard is the MP3 audio format . The standard
consists of the following five Parts 1. Systems
(storage and synchronization of video, audio, and
other data together) 2. Video (compressed video
content) 3. Audio (compressed audio content) 4.
Conformance testing 5. reference software
25
MPEG-2

• was designed for coding interlaced images at
  transmission rates above 4 million bits per
  second.
• MPEG 2 can be used on HD-DVD and blue ray disc.
• handles 5 audio channels,
• Covers wider range of frame sizes (HDTV).
• Provides resolution 720480 and 1280720 at 60
  fps with full CD quality audio used by DVD-ROM.
• MPEG-2 can compress 2 hours video into a few GHz.
• MPEG-2 is used for digital TV broadcast and DVD.
• An MPEG-2 is designed to offer higher quality
  than MPEG-1, at a higher bandwidth (between 4 and
  10 Mbit/s).
• The scheme is very similar to MPEG-1, and
  scalable.

26
MPEG-3

• Designed to handle HDTV signal in range 20 to 40
  Mbits/sec.
• HDTV-resolution is 1920 108030 Hz
• But MPEG-2 was fully capable of handling HDTV so
  MPEG -3 is no longer mentioned.

27
MPEG-4

• MPEG-4 is a collection of methods defining
  compression of audio and visual (AV) digital
  data.
• MPEG-4 absorbs many of the features of MPEG-1 and
  MPEG-2 and other related standards, Wavelength
  band MPEG-4 files are smaller than JPEG. so they
  transmit video and images over narrower bandwidth
  and can mix video with text graphics and 2D and
  3D animation layers.
• MPEG-4 provides a series of technolgies for
  developers for various service providers and end
  users.
• SP use for data transparency
• Helps end users with wide range of interaction
  with animated objects.
• MPEG-4 multiplexes and synchronizes data .
• Interaction with audio visual scene.

28
MPEG-7
MPEG-7 is a content representation standard for
information search. It is also titled Multimedia
Content Description Interface. It will define
the manner in which audiovisual materials can be
coded and classified so the materials can be
easily located using search engines just as
search engines are used to locate text-based
information . Music, art, line drawings, photos,
and videos are examples of the kinds of materials
that will become searchable based on descriptive
language defined by MPEG-7. Provide a fast
and efficient searching, filtering and content
identification method. Describe main issues
about the content (low-level characteristics,
structure, models, collections, etc.).
Index a big range of applications.
Audiovisual information that MPEG-7 deals is
Audio, voice, video, images, graphs and 3D
models Inform about how objects are
combined in a scene. Independence between
description and the information itself.
29
MPEG-7 applications

• Digital library Image/video catalogue,
  musical dictionary.
• Multimedia directory services e.g. yellow
  pages.
• Broadcast media selection Radio channel,
  TV channel.
• Multimedia editing Personalized electronic
  news service, media authoring.
• Security services Traffic control,
  production chains...
• E-business Searching process of products.
• Cultural services Art-galleries,
  museums...
• Educational applications.
• Biomedical applications.

30
(No Transcript)
31
TIFF(Tagged Image File Format)

• Tagged Image File Format (abbreviated TIFF) is a
  file format for storing images, including
  photographs and line art
• TIFF is a flexible, adaptable file format for
  handling images and data within a single file, by
  including the header tag.
• TIFF file using lossless compression (or none)
  may be edited and re-saved without losing image
  quality
• The TIFF format is the standard in document
  imaging and document management systems
• The TIFF format can save multi-page documents to
  a single TIFF file rather than a series of files
  for each scanned page. Multi-page support and 2D
  compression of bitonal images.

32
TIFF format
33
JPEG

• commonly used method of compression for
  photographic images.
• JPEG typically achieves 101 compression with
  little perceptible loss in image quality.
• JPEG compression is used in a number of image
  file formats. JPEG/Exif is the most common image
  format used by digital cameras and other
  photographic image capture devices
• it is the most common format for storing and
  transmitting photographic images on the World
  Wide Web. These format variations are often not
  distinguished, and are simply called JPEG.

34

• Format independent of frame size
• different data rates
• synchronization of different streams
  (audio, video)
• SW coding or HW coding
• open systems should be allowed.
• During the retrieval applications for human -gt
  database interaction such as
• Fast forward and backward data retrieval
• Random access
• Decomposition of images, video or audio
  independently should be possible.

35
GIF Graphics Interchange Format

• The Graphics Interchange Format (GIF) is a bitmap
  image format
• The format supports up to 8 bits per pixel,
  allowing a single image to reference a palette of
  up to 256 distinct colors chosen from the 24-bit
  RGB color space. It also supports animations
• GIF images are compressed using the
  Lempel-Ziv-Welch (LZW) lossless data compression
  technique to reduce the file size without
  degrading the visual quality.

36

• GIFs can also be used to store low-color sprite
  data for games.
• GIFs can be used for small animations and
  low-resolution film clips.
• In view of the general limitation on the GIF
  image palette to 256 colors, it is not usually
  used as a format for digital photography
• The PNG format is a popular alternative to GIF
  images since it uses better compression
  techniques and does not have a limit of 256
  colors, but PNGs do not support animations.

37
Despite its other limitations, fluid 3D animation
is possible when using the GIF format, as this
animation of Newton's Cradle demonstrates.
38
Other Formats

• PNG (Portable Network Graphic) A higher-quality
  replacement for the GIF format.
• PNG's compression is among the best that can be
  had without losing image information.
• PNG supports three main image types truecolor,
  grayscale and palette-based ("8-bit").
• PNG doesnot support animation at all.
• It has greater compression than GIF.

39
(No Transcript)
40
PDF

• PDF (Portable Document Format) provides a
  convenient way to view and print images at a high
  resolution
• PDF lets us to capture and view robust
  information from application on any computer
  system.
• Multiplatform PDF are viewable and printable on
  any platform.
• Extensible-1800 vendors world wide offer PDF
  based solutions.
• Trusted realiable. preserve source file
  information, text, drawing etc regardless of
  applications.
• Digital sign or password pdfdoc created with
  acrobat SW,
• Searchable- Text search features.

41
Structuring Metadata

• Metadata schemes (also called schema) are sets of
  metadata elements designed for a specific
  purpose, such as describing a particular type of
  information resource.
• The definition or meaning of the elements
  themselves is known as the semantics of the
  scheme.
• ASCII Text
• SGML (Standard Generalized Markup Language)
• HTML (HyperText Markup Language)
• XML
• XHTML (Extensible HyperText Markup Language)
• MARC (The MAchine Readable Cataloginge)