Ch 2: Perception and Cognition

1
Ch 2 Perception and Cognition

• Audition (sound perception)
• Physical properties, describing sounds
• Physiology, perception
• Visual perception
• Physical properties, Describing light and images
  and scenes
• Physiology, perception
• Cognition
• Interpretation of perceptions, understanding,
  acting on these interpretations
• Interactions between perception cognition
• Perception influences cognition
• Cognition influences perception

2
Sound

• For humans, visual information is often
  considered primary
• Acoustic information is still highly relevant
• Sounds generated by physical vibrations
• Vibrations induce waves
• e.g., compression/rarefaction in the air
• Sound reception physiological/psychological
• Rapid pressure changes detected by ears
• Also possible bone conduction
• Sound received ? sound produced ? sound perceived

3
Waveform Terminology - 1

• Cycle
• one complete unit of a periodic waveform

Example A sine wave
Interval between similar points is a cycle
4
Waveform Terminology - 2

• Hz hertz unit of frequency
• Number of times an event occurs per second
• E.g., number of cycles per second in a waveform
• Human sound perceptual range
• Around 20 20,000 Hz
• Infants and children have widest range

5
Example

• What is the frequency of this waveform?

6
Trick question!

• You need to have a scale on the x-axis in seconds
  to determine the frequency!

0 sec
1 sec

• What is the frequency?

7
Waveform Terminology - 3

• Amplitude
• height of a waveform, from 0 on y axis
• Units of amplitude vary with type of energy in
  waveform
• Sound wave amplitude directly relates to loudness
  of sound
• Loudness measured in dB, Decibels

8
Waveform Terminology - 4

• Phase relative timing of identical waves (same
  amplitude wavelength)
• In Phase Same timing

Wave 1
Wave 2
9
Waveform Terminology - 5

• Phase relative timing of identical waves (same
  amplitude wavelength)
• Out of Phase Different timing

Wave 1
Wave 2
10
Waveform Terminology - 6

• Harmonics
• Waveforms with frequencies that are integral
  multiples of others

11
Waveform interaction

• When different waveforms are produced at the same
  time
• E.g., two sounds produced at the same time
• The waveforms interact and combine
• Observed result
• Sound is different
• Combined result of multiple waveforms
• http//www.cprince.com/courses/cs3121fall03/lectur
  es/Ch2waveform-math.xls

12
Waveform math

• Waveforms can be described by equations
• y f(x)
• x gives time, along the x-axis
• y is amplitude
• Two waveforms
• y1 f1(x)
• y2 f2(x)
• When the two waveforms occur together, they
  produce a new waveform
• Mathematically, describe this by summing
• ynew f1(x) f2(x)

13
Example Waveform

• y f(x)
• sin(x)

14
y1 sin(x)
y2 2sin(x)
sin(x) 2sin(x)
15
sin(x) .951
Examplex1.27
cos(x) .309
sin(x) cos(x) 1.26
16
Phase Cancellation

• Two waves with same frequency and amplitude
• Propagated at same time and from same place
• One waveform is 1/2 cycle (180 degrees) out of
  phase with other
• Get Phase Cancellation
• Waveforms cancel each other
• No sound

17
sin(x)
sin(xpi)
sin(x) sin(x pi)
18
Frequency-based Analysis of Sounds

• Any sound can be exactly described as the sum of
  sine wave frequency components
• Sine wave components
• Also called Fourier Analysis
• Fast Fourier Transform is one algorithm that can
  perform this analysis
• Not necessarily related to human perceptual
  analysis

19
200 hz sine wave
Raven sound analysis software from Cornell
University
20
300 hz sine wave
21
100 hz 1000 hz sine wave
22
Speech
23
Intensity of Sound

• Intensity (amplitude)
• Physical measurement units Decibels, acoustic
  watts
• Decibels (dB)
• Logarithmic scale
• 0 dB Defined as human hearing threshold
• 3 db change typically required to detect
  difference
• Corresponds to at least a doubling of power
  (amplitude)
• What we perceive as a doubling of loudness
  corresponds to approximately an increse in
  amplitude by an order of magnitude
• Human hearing is nonlinear it responds
  logarithmically

24
Pitch Perceptual Measurement

• pitch is the psychological correlate of
  frequency (p. 401, Gelfand, 1990)
• We perceive some features of physical sound
  properties
• There is not always a simple relationship
• Pitch
• Related to frequency
• Generally
• Higher frequency gt higher pitch
• Lower frequency gt lower pitch
• However
• Doubling the frequency does not always double the
  pitch
• Intensity (amplitude) change can change pitch

Gelfand, S. A. (1990). Hearing An Introduction
to Pyschological and Physiological Acoustics (2nd
Ed). New York Marcel Dekker.
25
Loudness Perceptual Measurement

• Perception of the intensity of a sound
• Generally, increasing intensity is perceived as
  increasing loudness
• Related to both amplitude and frequency
• E.g., increasing the frequency of a sound may
  increase its loudness

26
500 Hz and 3000 Hz example
27
Sound Perception

• Sound similarities
• Example We interpret speech sounds as the same
  despite large variations
• volume, pitch, noise, accent
• Sound localization
• Determining the location of a sound source
• Cues or information used
• Effects due to pinnae (outer ear)
• Ear disparity effects (Interaural temporal
  differences)
• Synchronization between sound and visual
  information
• Identification
• Identifying the source of a sound (e.g., speech
  sounds you hear over a telephone)
• People use various cues to identify sounds
• Context, physical attributes of sound, subjective
  attributes

28
Sounds Influences Between Perception Cognition

• Habituation
• Brains mainly attend to change
• An influence of perception on cognition
• In general, the longer one kind of sound stimulus
  occurs, the less we attend to that stimuli
• I.e., the less interesting it becomes
• Context effects
• Example
• Word perception can be influenced by context
• An influence of perception by cognition
• Isolated word recognition tends to be more
  difficult than recognizing a word in a sentence
  (context)

29
Physical Properties of Light

• Intensity- visible light (unit candela)
• Power per unit area
• Strength decreases as the inverse square of
  distance
• E.g.,
• At one foot, say intensity is I
• at two feet, intensity ? I/22
• at three feet, intensity ? I/32
• Intensity is diminished by passing through
  materials
• Radiance all electromagnetic radiation
• unit watt
• total amount of energy (e.g., light plus heat)
  emitted from a source

30
Psychological Properties

• Luminance
• measure of light strength perceived by the human
  eye (unit lumen)
• Brightness subjective measure of relative
  brightness

31
Color and Color Sensation

• Object color
• When object is not emitting light, we see color
  of the reflected light

White light
Subtractive process Incoming absorbed
reflected
object
Wavelengths not absorbed
32
Subtractive PrincipleReflected Light

• Objects, pigments, etc. absorb, or subtract
  certain wavelengths of light
• Various methods used to subtractively create
  object colors
• For example
• Pigments mixed to form a new pigment
• Combine subtractive qualities of pigments
• Mix pigment A (absorbs everything but red), with
  pigment B (absorbs red) should get black

33
Additive Principle Emitted Light

• Colored lights mix, combine to form new colors
• E.g., computer monitor
• May combine red, green, blue (RGB) dots to form
  color needed in particular situation
• Other examples
• TV, color photographic enlarger

34
Trichromatic Vision

• Human retina Different types of photoreceptive
  cells
• Rods low light/motion
• Cones different types of photopigments
• Absorbs 435 nm light (blue)
• Absorbs 540 nm light (green)
• Absorbs 656 nm light (yellow)
• Generation of color
• Perception of any color can be created by mixing,
  in varying proportions three primary colors
  (red, green, blue)

35
Perception of ColorAlso Depends On

• Physiology
• Cornea is yellow (adaptive for UV light)
• Filters out shorter wavelengths (green, blue,
  violet)
• Allows red through
• Change or stability of light
• (1) Rapid pulse or change in brightness
• Causes perception of different colors
• (2) Retinal cells adapt to common
  colors/intensities
• And can also become fatigued
• Color context
• Affective aspects
• Colors tend to be associated with emotions

36
Opponent coding

• Complementary colors apparently arise from the
  opponent coding of the visual system. After
  staring at a single color for a prolonged period,
  the retinal cells that contribute to the
  definition of the color become fatigued. When
  sight is directed to a white page, these cells
  briefly can't contribute to the definition of
  white, leaving "white" to be represented by the
  unfatigued cells on the opposite side of the
  opponent contrasts. (http//www.handprint.com/)

37
Describing Color

• Various color spaces
• Color space system for precisely specifying
  colors
• Some color spaces
• RGB
• Red, green, blue
• HSB
• Hue, saturation, brightness
• YUV
• Luminance and chrominance coding
• In general, we need three parameters (e.g., three
  colors) in order to specify a color of visible
  light

38
HSB Color Space

• Transformation of the RGB color space to terms
  more natural to an artist
• Hue Wavelength of light
• Saturation Amount of the hue
• degree to which a hue differs from a neutral gray
• 0 - no color saturation- white
• 100, full saturation- the pure hue
• Brightness Intensity of the color
• 0 - black
• 100 - as bright as possible

http//escience.anu.edu.au/lecture/cg/Color/HSV_HL
S.en.html
39
YUV Color Space

• Human perception is more sensitive to brightness
  than any color information
• So, it can be useful to code color in a way that
  separates between brightness and color
• Y luminance channel
• UV chrominance channels
• Typically, Y is coded using more information
  (e.g., bits) than U and V, because brightness
  errors are more noticeable than color errors

(Steinmetz Nahrsted, 1995)
40
Vision Perception Cognition

• Generally, influences in both directions
• Cognition-gtperception
• E.g., Our expectations influence what we see
• Perception-gtcognition
• We think about what we see
• Sometimes, cognition cant influence perception

41
Depth Perception

• What cues enable depth to be perceived?
• Stereovision differences between images
• Field of vision 180 degrees
• Field in which depth is perceived 90 degrees
• Other cues to depth
• E.g., texture gradients, occlusion (interposition)

42
Occlusion
43
Motion

• Real vs. apparent motion
• Real An object moving in the world
• Apparent Some kind of illusion of motion
• e.g.
• A video image on the TV or computer
• Animation
• Critical flicker fusion frequency (CFFF) about
  15 hz
• Real vs. apparent causality Perception of
  physical causality
• Michotte How do humans perceive that one event
  has physically caused another?
• Launching
• Entraining
• Object B starts as soon as object A passes by
• Demo http//www.carleton.ca/warrent/210/michotte
  /michotte.html

44
Camera vs. Eye

• Typical Camera
• Passive optical system
• Visual Perception
• Popular or traditional view
• rays of light that constitute images that
  regarded as faithful representations of the
  external world (p. 68, main text)
• But what about levels of illumination, shadows,
  occlusion, dynamics of motion?
• Watt (1991) We would be nearer the mark if we
  were to look to the artist, painting first with
  his coarse brush and then building within and
  over these broad strokes with finer details The
  artist is an active device

45
Active Perception

• The perceiver is considered to act as an
  intuitive i.e., naïve scientist, making and
  testing hypotheses about the world on the basis
  of inadequate or distorted sensory evidence
  (Gordon, 1989, p. 12)
• Sensory information is ambiguous
• Multiple interpretations are the rule
• We can use actions to disambiguate sensation
• E.g., poking an object to separate foreground
  from background
• We are trying to make use of active perception in
  multimedia

46
Examples from MIT

• Metta Fitzpatrick (2002)
• Machine vision Using poking to visually segment
  an object from background

47
Simons Chabris (1999)

• Instructions
• Look at one of the teams of players and count the
  number of ball passes they make
• Please dont make any sounds!
• http//www.wjh.harvard.edu/viscog/lab/demos.html

48
Simons, Reimer, Franconeri (2000)

• Instructions
• Look at the following image and see if you can
  detect any changes.
• http//www.wjh.harvard.edu/viscog/lab/demos.html

49
Inattentional Blindness

• The last two slides were examples of
  inattentional blindness
• At times, we only perceive information that we
  are actively attending to
• Message for multimedia
• We need to make sure we focus the user on what we
  want them to learn and know
• We cant always be assured that the user will see
  (or, presumably, hear) what we want them to see
  (or hear)!