Multimedia%20Systems%20

1
Multimedia Systems Interfaces

• Karrie G. KarahaliosSpring 2007

2
Perception

• Review
• Expectations
• Perception
• Homework

3
Color and Visual System

• Color refers to how we perceive a narrow band of
  electromagnetic energy
• source, object, observer
• Visual system transforms light energy into
  sensory experience of sight

4
Human Visual System

• Eyes, optic nerve, parts of the brain
• Transforms electromagnetic energy

5
Human Visual System

• Formation
• cornea, sclera, pupil,iris, lens, retina, fovea
• Transduction
• retina, rods, and cones
• Processing
• optic nerve, brain

6
Image Formation

• Cornea and sclera
• Pupil
• Iris
• Lens
• Retina
• Fovea

Sclera
Retina
Cornea
Fovea
Lens
Pupil
Iris
7
The Cornea

• Part of sclera hard white part of the eye
• Transparent part at front of eye
• Allows light to enter, refraction occurs

8
The Pupil and Iris

• Controls amount of light passing through
• diameter varies in response to light
• Iris controls the diameter of the pupil
• gives eye its color

9
The Lens

• Focuses light on the retina using refraction
• Changes shape to provide focus
• spherical forcloser objects
• flat for far objects
• accommodation

10
The Retina and Fovea

• Retina has photosensitive receptors at back of
  eye
• Fovea is small, dense region of receptors
• only cones (no rods)
• gives visual acuity
• Outside fovea
• fewer receptors overall
• larger proportion of rods

11
The Transduction

• Transform light to neural impulses
• Receptors signal bipolar cells
• Bipolar cells signal ganglion cells
• Axons in the ganglion cells form optic nerve

Rods
Bipolar cells
Ganglion
Cones
Optic nerve
12
(No Transcript)
13
(No Transcript)
14
Rods and Cones
Cones
Rods

• Contain photo-pigment
• Respond to low energy
• Enhance sensitivity
• Concentrated in retina, but outside of fovea
• One type, sensitive to grayscale changes

• Contain photo-pigment
• Respond to high energy
• Enhance perception
• Concentrated in fovea, exist sparsely in retina
• Three types, sensitive to different wavelengths

15
Rod and Cone Destiny
120 million rods
6-7 million cones
From http//hyperphysics.phy-astr.gsu.edu/hbase/vi
sion/rodcone.html
16
Tri-stimulus Theory

• 3 types of cones (6 to 7 million of them)
• Red (64), Green (32), Blue (2)
• Each type most responsive to a narrow band
• red and green absorb most energy, blue the least
• Light stimulates each set of cones differently,
  and the ratios produce sensation of color

17
Tri-stimulus Theory
18
Opponent-Color Theory

• Visual system contains two types of
  color-sensitive units
• red / green blue / yellow
• Each component in a unit responds opposite the
  other component
• e.g., if red-green responds to red, then green
  is inhibited
• Explains concept of after images

19
Visual System Facts

• Distinguish hundreds of thousands of colors
• more sensitive to brightness
• Can distinguish about 28 fully saturated hues
• less sensitive to hue changes in less saturated
  colors
• Can distinguish about 23 levels of saturation for
  fixed hue and lightness
• 10 times less sensitive to blue than red or green
• it absorbs less energy in the blue range

20
Physical Properties of Color

• Dominant wavelength
• electromagnetic waves
• 400nm (violet) to 700nm (red)
• Excitation purity
• Luminance

21
Spectral Distribution

• Dominant wavelength
• spike in power (e2)
• white light is uniform energy distribution
• Excitation purity
• ratio between e2 / e1

e2
Energy distribution
e1
Wavelength
22
Perceptual Properties of Color

• Hue
• distinguishes named colors, e.g., RGB
• dominant wavelength of the light
• Saturation
• how far color is from a gray of equal intensity
• Brightness (lightness)
• perceived intensity

23
Color Perception

• Hue
• distinguishes named colors, e.g., RGB
• dominant wavelength of the light
• Saturation
• how far color is from a gray of equal intensity
• Brightness (lightness)
• perceived intensity

Tints
Purecolors
White
Tones
Grays
Shades
Black
24
CIE

• Standard reference

25
Color Models
Subtractive (CMY)
Additive (RGB)
Applies to reflected light (printed images,
paints, etc)
Applies to light-emittingsources (TVs, monitors,
etc)
26
HSV (aka HSB)

• User-oriented, based on use of tints, shades, and
  tones

27
RGB

• Based on the fact that the human visual system
  maintains three types of cones (RGB cones)
• Different weightings produce different colors

Green(0,1,0)
Yellow(1,1, 0)
White(1,1,1)
Cyan(0,1,1)
Black(0,0,0)
Red(1,0,0)
Blue(0,0,1)
Magenta(1,0,1)
28
YIQ (aka YUV)

• Used in US television broadcasting
• Recoding of RGB for efficiency and compatibility
  with black-and-white TV
• Y is luminance (not yellow!)
• I and Q is chromaticity

29
CMYK

• Subtractive color model
• used for printing, painting, etc.
• CMY are the complements of RGB
• two complementary colors gives a primary

_

30
Gestalt
31
Auditory Perception
32
Waves

• Periodic disturbance that travels through a
  medium (e.g. air or water)
• Transport energy
• Transverse or longitudinal
• Electromechanical or mechanical

33
Sound

• A longitudinal, mechanical wave
• caused by a vibrating source
• Pack molecules at different densities
• cause small changes in pressure
• Model pressure differences as sine waves

34
Volume and Pressure
35
Auditory System

• Ears, parts of brain, and neural pathways
• Changes in pressure move hair-like fibers within
  the inner ear
• Movements result in electrical impulses sent to
  the brain

36
Process of Hearing (Transduction)
37
Frequency (temporal) Theory

• Periodic stimulation of membrane matches
  frequency of sound
• one electrical impulse at every peak
• maps time differences of pulses to pitch
• Firing rate of neurons far below frequencies that
  a person can hear
• Volley theory groups of neurons fire in
  well-coordinated sequence

38
Place Theory

• Waves move down basilar membrane
• stimulation increases, peaks, and quickly tapers
• location of peak depends on frequency of the
  sound, lower frequencies being further away

39
Physical Dimensions

• Amplitude
• height of a cycle
• relates to loudness
• Wavelength (w)
• distance between peaks
• Frequency ( ? )
• cycles per second
• relates to pitch
• ?w velocity
• Most sounds mix many frequencies amplitudes

Sound is repetitive changesin air pressure over
time
40
Psychological Dimensions

• Loudness
• higher amplitude results in louder sounds
• measured in decibels (db), 0 db represents
  hearing threshold
• Pitch
• higher frequencies perceived as higher pitch
• hear sounds in 20 Hz to 20,000 Hz range

41
Psychological Dimensions

• Timbre (tam-bre)
• complex patterns added to the lowest, or
  fundamental, frequency of a sound, referred to as
  spectra
• spectra enable us to distinguish musical
  instruments
• Multiples of fundamental frequency give music
• Multiples of unrelated frequencies give noise

42
Sound Intensity

• Intensity (I) of a wave is the rate at which
  sound energy flows through a unit area (A)
  perpendicular to the direction of travel
• P measured in watts (W), A measured in m2
• Threshold of hearing is at 10-12 W/m2
• Threshold of pain is at 1 W/m2

43
Decibel Scale

• Describes intensity relative to threshold of
  hearing based on multiples of 10

44
Decibels of Everyday Sounds
Sound Decibels
Rustling leaves 10
Whisper 30
Ambient office noise 45
Conversation 60
Auto traffic 80
Concert 120
Jet motor 140
Spacecraft launch 180
45
Loudness from Multiple Sources

• Use energy combination equation
• where L1, L2, , Ln are in dB

46
Exercises

• Show that the threshold of hearing is at 0 dB
• Show that the threshold of pain is at 120 dB
• Suppose an electric fan produces an intensity of
  40 dB. How many times more intense is the sound
  of a conversation if it produces an intensity of
  60 dB?
• One guitar produces 45 dB while another produces
  50 dB. What is the dB reading when both are
  played?
• If you double the physical intensity of a sound,
  how many more decibels is the resulting sound?

47
Loudness and Pitch

• More sensitive to loudness at mid frequencies
  than at other frequencies
• intermediate frequencies at 500hz, 5000hz
• Perceived loudness of a sound changes based on
  the frequency of that sound
• basilar membrane reacts more to intermediate
  frequencies than other frequencies

48
Fletcher-Munson Contours
Each contour represents an equal perceived sound
49
Masking

• Perception of one sound interferes with another
• Frequency masking
• Temporal masking

50
Frequency Masking

• Louder, lower frequency sounds tend to mask
  weaker, higher frequency sounds

From http//www.cs.sfu.ca/CourseCentral/365/
51
Frequency Masking

• Louder, lower frequency sounds tend to mask
  weaker, higher frequency sounds

52
Frequency Masking

• Louder, lower frequency sounds tend to mask
  weaker, higher frequency sounds

53
Temporal Masking

• When exposed to a loud sound, the human ear
  contracts slightly to protect delicate structures
• Causes louder sounds to overpower weaker sounds
  just before and just after it

From http//www.cs.sfu.ca/CourseCentral/365/
54
Combined Masking
From http//www.cs.sfu.ca/CourseCentral/365/
55
Localization of Sound

• Localization occurs because
• sound reaches one ear before the other
• the head creates a sound shadow that decreases
  intensity of the sound to far ear

56
Assignment

• Read pamphlets by Andries Van Dam
• Color
• Illumination
• Intensity Demo

57
Density Demonstration
58
(No Transcript)