Multimedia communications EG-371Dr Matt Roach

1
Multimedia CommunicationsEG 371 and EG 348

• Dr Matthew Roach
• dr.matt.roach_at_iee.org
• Lecture 1
• Overview

2
Recommended texts

• F. Fluckiger, Understanding Networked Multimedia,
  Prentice Hall, 1995.
• R.A. Earnshaw and J.A. Vince, Multimedia Systems
  and Applications, Academic Press 1995.
• J. Ozer, Video Compression for Multimedia, AP
  Professional, 1995.
• F. Halsall Multimedia communications
  applications, networks, protocols and standards,
  Addison-Wesley, 2001.

3
Course

• Course notes online
• Read up
• Make own notes
• http//galilee.swan.ac.uk/
• Publications teaching EE371
• 10 lectures
• Engineering E, 1100
• 2 Tutorials
• Worksheets, last years exam questions

4
Course overview

• Visual signals
• Image definitions
• Image formats
• Video formats
• image information
• image compression

5
What is multimedia

• Media transferred over the net
• Text
• Formatted, unformatted
• Images
• Computer generated, digitised photographs
• Audio
• Speech, sound
• Video
• All of the above

6
Multimedia - video

• Digital Video
• Visual signal
• Number of frames per second
• A frame is a static image
• Spatial resolution
• Colour resolution
• Acoustic signal
• Sampling rate
• Quantisation of the amplitude
• Meta-data
• Text descriptions

7
Digital video signals

• What is a signal
• Functions
• Scalar
• Vector
• Video capture
• Video storage

8
Fundamentals of compression

• Data information
• Redundancy
• Compression ratio
• Types of redundancy
• Coding
• Inter-pixel
• Psyco-visual
• Data compression

9
Fundamentals of compression

• Design concerns
• Trade-off
• quality loss Vs c compression ratio
• Measuring quality loss
• Fidelity Criteria

10
Coding compression

• Histograms
• Entropy encoding
• Run length encoding
• Huffman encoding
• Differential encoding

11
JPEG

• Static image compression standard
• Image/block preparation
• Forward DCT
• Quantisation
• Entropy encoding
• Frame building

12
Tutorials Exam questions

• Any questions?

13
Signals and Functions

• What is a signal
• Signal function (variable with physical
  meaning)
• one-dimensional (e.g. dependent on time)
• two-dimensional (e.g. images dependent on two
  co-ordinates in a plane)
• three-dimensional (e.g. describing an object in
  space)
• higher-dimensional
• Scalar functions
• sufficient to describe a monochromatic visual
  signals black white video and sound.
• Vector functions
• represent colour images - three component colours

14
Visual Functions

• image - continuous function of a number of
  variables
• Co-ordinates x, y in a spatial plane
• for image sequences - variable (time) t
• Image function value brightness at image points
• other physical quantities
• temperature, pressure distribution, distance from
  the observer
• Image on the human eye retina / TV camera sensor
  - intrinsically 2D
• 2D image using brightness points intensity
  image
• Mapping 3D real world -gt 2D image
• 2D intensity image perspective projection of
  the 3D scene
• information lost - transformation is not
  one-to-one
• geometric problem - information recovery
• understanding brightness info

15
Visual Acquisition Manipulation

• Analogue camera
• frame grabber
• video capture card
• Digital camera / video recorder
• Capture rate ? 30 frames / second
• Human Visual system (HVS) persistence of vision
  
• Computer, digitised image, software (example c)
• f(x,y) ? define M 128
• define N 128
• unsigned char fNM
• 2D array of size NM
• Each element contains an intensity value

16
Image definition

• Image definition
• A 2D function obtained by sensing a scene
• F(x,y), F(x1,x2), F(x)
• F - intensity, grey level
• x,y - spatial co-ordinates
• No. of grey levels, L 2B
• B no. of bits

Why?
17
Brightness and 2D images

• Brightness dependent several factors
• object surface reflectance properties
• surface material, microstructure and marking
• illumination properties
• object surface orientation with respect to a
  viewer and light source
• Some Scientific / technical disciplines work with
  2D images directly
• image of flat specimen viewed by a microscope
  with transparent illumination
• character drawn on a sheet of paper
• image of a fingerprint

18
Monochromatic images

• Image processing - static images - time t is
  constant
• Monochromatic static image - continuous image
  function f(x,y)
• arguments - two co-ordinates (x,y)
• Digital image functions - represented by matrices
• co-ordinates integer numbers
• Cartesian (horizontal x axis, vertical y axis)
• OR (row, column) matrices
• Monochromatic image function range
• lowest value - black
• highest value - white
• Limited brightness values gray levels

19
Chromatic images

• Colour
• Represented by vector not scalar
• Red, Green, Blue (RGB)
• Hue, Saturation, Value (HSV)
• luminance, chrominance (Yuv , Luv)

S0
Green
Hue degrees Red, 0 deg Green 120 deg Blue 240 deg
Red
Green
V0
20
Use of colour space
21
Visual signal quality

• Quality of digital visual signal proportional to
• spatial resolution
• proximity of image samples in image plane
• spectral resolution
• bandwidth of light frequencies captured by sensor
• radiometric resolution
• number of distinguishable gray levels
• time resolution
• interval between time samples at which images
  captured

22
Image summary

• F(xi,yj)
• i 0 --gt N-1
• j 0 --gt M-1
• NM spatial resolution, size of image
• L intensity levels, grey levels
• B no. of bits

23
Digital Image Storage

• Stored in two parts
• header
• width, height cookie.
• Cookie is an indicator of what type of image file
• data
• uncompressed, compressed, ascii, binary.
• File types
• JPEG, BMP, PPM.

24
PPM, Portable Pixel Map

• Cookie
• Px
• Where x is
• 1 - (ascii) binary image (black white, 0 1)
• 2 - (ascii) grey-scale image (monochromic)
• 3 - (ascii) colour (RGB)
• 4 - (binary) binary image
• 5 - (binary) grey-scale image (monochromatic)
• 6 - (binary) colour (RGB)

25
PPM example

• PPM colour file RGB
• P3
• feep.ppm
• 4 4
• 15
• 0 0 0 0 0 0 0 0 0 15 0 15
• 0 0 0 0 15 7 0 0 0 0 0 0
• 0 0 0 0 0 0 0 15 7 0 0 0
• 15 0 15 0 0 0 0 0 0 0 0 0