Cosc 6326/Psych6750X

1
Cosc 6326/Psych6750X

• Vision and Visual Displays

2

• Some theories of perception
• Empiricism vs Nativism
• Gestalt
• Ecological
• Constructionist
• Active perception

3
http//www.nipissingu.ca/stange/courses/p2255/2255
-images/gestalt.jpg
4

• Is vision inverse graphics?
• Techniques to study vision include
• Philosophical, Introspective Anaylsis
• Psychophysics
• Clinical Deficits
• Electrophysiology
• Anatomical studies
• Imaging
• Computational Analysis

5
Levels of Processing

• Early vision
• low level processing related to extracting image
  and surface properties such as luminance changes,
  depth and motion
• Intermediate-level vision
• Grouping operations, smoothing, cue integration,
  computation of shape, colour, contour

6

• High-level vision
• task based object recognition, relations between
  objects, selection for manipulation/navigation

7

• Marr 1982 claimed that a visual system can be
  described at three levels of abstraction
• computational
• algorithmic
• implementation

8
Structure of the Eye

• About 70 of refractive power in cornea
• Lens fine tunes focus depending on distance and
  pupil size (accommodation)
• Pupil size adjusted to trade amount of light with
  depth of field

9
Structure of the Eye

• Retina contains sensory receptors
• In humans, receptor density is non-uniform
• Highest density in centre of foveal pit
  decreases with eccentricity
• Visual axis joins point of fixation with fovea
• offset from optic axis

10
(No Transcript)
11
Functional Organisation of the Retina

• Two types of photoreceptors
• Rods
• high sensitivity
• exclusively peripheral distribution
• broad spectral tuning
• retina contains gt108 rods

12
Functional Organisation of the Retina

• Cones
• 3 wavelength selective types
• Long (? 40, 565nm peak)
• Medium (? 40, 535nm peak)
• Short (? 10, 450nm peak)
• ? 5 Million, highest concentration in fovea
  (gt6000 in central 1)

13
(No Transcript)
14
Spectral Sensitivities and Colour Vision Variants
15
Limits on Vision

• Spectral sensitivity
• 400-700 nm range
• 3 cone types
• Precise discrimination but no resolution
• Sensitivity to intensity
• Sensitivity to spatial variation
• Sensitivity to temporal variation
• Field of view

16
Limits on sensitivity to intensity

• Phototopic range
• day time vision, colour sensitivity, high
  resolution, cone-mediated
• Scotopic
• low light vision, achromatic, rod-mediated, more
  sensitive but low spatial resolution
• Mesatopic
• transition range (e.g. moonlight)

17
(No Transcript)
18
Dynamic Range

• Dynamic range of eye is about three log units
• Range of visible stimuli is several log units, we
  rely on
• adaptation
• pupil size changes
• rod and cone sensitivity ranges

19
Dynamic Range

• Dynamic range of most displays is limited to a
  small fraction of natural range (1001 versus up
  to 10,0001 in a natural scene)
• Issues are both display capability and image
  generation
• Tone mapping is one technique that uses
  non-linear luminance mapping to distort image
  increase range of visibility
• Some true HDR displays in development (including
  one at York)

20
Tone Mapping
Larson, Siggraph 1997
21
Retinal Microcircuits

• Receptive field of a neuron refers to the region
  of the retinal that contributes (excites or
  inhibits) to its firing rate
• Ganglion receptive fields have a centre-surround
  organisation
• on-centre light in centre, inhibited by light in
  surround
• off-centre excited by its surround, inhibited by
  light in centre

22
Retinal Microcircuits
23
from Palmer 1999
24

• firing rate cannot be negative leads to
  rectification of response
• off-channels complement response of on-channels
• centre-surround organisation gives sensitivity to
  change in luminance

25
http//www.cnl.salk.edu/thomas/coce.html
26

• Contrast dependent processing occurs at higher
  levels of the brain as well

27
Visual Pathways

• principally lateral geniculate nucleus (LGN) then
  cortex
• also important sub-cortical pathways
• Accessory optic tract
• Pulvinar, superior colliculus
• massive feedback as well as feedforward pathways

from Howard Rogers 2001
28
Visual Areas
29

• neurons usually sensitive to more than one
  feature
• primary visual cortex
• spatio-topically mapped
• has columnar organization (hyper-columns) for
• orientation
• direction of motion
• eye dominance columns

30

• Function of oriented receptive fields in V1?
• edge and bar detectors (black inhibitory, white
  excitatory)?
• tuned to a variety of different orientations

Receptive Field
Stimulus eliciting best response
31

• Fourier analysis of image (Blakemore and
  Campbell)? Need to combine responses of receptors
  with similar receptive fields distributed over
  visual field
• spatial-temporal filtering

32

• Visual areas believed to be specialised for
  features, e.g.
• V1 oriented edges, colour
• MST, MT motion, optic flow
• V4 colour, form
• IT objects
• Well have time to look at only a small set of
  the functions vision supports

33
(No Transcript)
34

• 2 pathways hypothesis (seminar last Wednesday)
• parasol ganglion cells project to magnocellular
  layers of LGN
• high contrast sensitivity, larger receptive
  fields, insensitive to colour, short latency,
  sensitive to luminance changes
• presumed to form input to a motion and depth
  pathway (to MT, MST )

35

• midget ganglion cells project to parvocellular
  layers of LGN
• colour opponent cells, less contrast sensitivity,
  colour sensitive, high spatial resolution, low
  temporal resolution
• specialised pathways for form and colour (V4, IT
  )
• Livingston and Hubel (1988) have suggested there
  may be 4 physiologically distinct pathways
  colour, binocular vision, motion and form

36
(No Transcript)
37
Rolls and Deco 2002
38
Spatial Resolution

• Fovea
• Optics of eye blur image
• Act as spatial frequency filter and prevents
  aliasing
• Optics are diffraction limited for small pupil
  sizes
• Retina is spherical, thus resolution is expressed
  in degrees of visual angle

39

• best acuity in central fovea
• receptor spacing closely matched to optics
• grating 60 cycles per degree
• line separation 30 seconds of arc
• Periphery
• resolution limited by sampling density of
  receptors
• best resolution for scotopic (rod) vision is
  extrafoveal where rod density increases

40
Contrast Sensitivity Function

• important caveat - acuity is for high contrast
  patterns under ideal conditions
• contrast required for an object to be visible
  depends on the pattern
• sensitivity falls at high and low spatial
  frequencies

41
Display Resolution

• resolution of displays usually measured in
  pixels/display or pixels/cm
• e.g. 30 cm wide, 1024x768 pixel display (30
  pixels per cm) would needs to be located at
• away for pixel pitch to subtend 30 seconds of
  arc
• effective resolution typically less than pixel
  pitch

42

• For full foveal acuity in a six walled CAVE how
  many pixels do we need?
• simplify to spherical surface (4p steradians)
  need about 5.9 x 108 pixels (Hopper, 2000) per
  eye
• typical 6 sided CAVE has 1280x1024x6 7.9x106
  pixels, a two-order of magnitude difference
• this is the key to the appeal of foveated displays

43
Field of View

• each has a visual field extending approx 56
  nasally and 95 temporally 190 total visual
  field
• extended to 290 with eye movements and nearly
  360 with head movement
• portion seen by both eyes is up to 114
  horizontally and 125 vertically

44
Field of view, resolution trade-offs for displays

• Instantaneous field of view (FOV) of a display is
  angle subtended by the image at the eye
• In HMDs FOV is restricted by size of the display
  and optical aberrations/distortions, which
  increase with eyepiece FOV
• typical values 20-100 horizontal FOV

45

• inherent trade-off between FOV and resolution for
  a fixed number of pixels
• many HMD systems have very poor resolution. For
  example video see-through HMD (640x480)
• 50 horizontal FOV, 12 pixels per degree
• 20 horizontal FOV, 32 pixels per degree

46

• large FOV easier in optical see-through HMDs (at
  least for see-through portion)
• tiling of multiple sub-displays has been proposed
  for large FOV HMDS
• resolution/FOV tradeoff also limits large format
  projection displays and tiling has been also used
  

47
Hyperacuities

• Hyperacuity (Westheimer McKee paper)
• ability to discriminate relations between
  features that are finer than the resolving power
  of the visual system or even photoreceptor
  spacing
• e.g. vernier acuity. Which line is higher? Foveal
  threshold is on the order of 1-5 seconds of arc
• implies interpolation ( population coding)

48
(No Transcript)
49

• other hyperacuities
• stereopsis
• curvature
• bisection
• orientation
• even colour discrimination (three wavelengths
  detected but can discriminate millions of colours)

50

• closely related to anti-aliasing in graphics.
• but need to trade off effective resolution for
  sub-pixel positioning
• displays that provide hyper-acuity level
  positioning without sub-pixel techniques are
  usually not feasible, e.g. 5 arcsec pixels
• 21600x16200 pixels for a 30 cm wide display at
  57cm
• 21 trillion pixels in a spherical cave

51

• In hyper-acuity we are more sensitive to small
  changes in relations between elements
• in VR and AR we measure pose of the head and
  generate appropriate images
• We are much more sensitive to relative
  mis-registration between real and synthetic
  imagery in AR than absolute misalignment in VR
• distortion and aberration more apparent in
  see-through displays than in VR displays

52
Temporal Vision

• critical fusion frequency temporal alternations
  above about 60 Hz are fused and perceived as
  steady illumination
• depends on contrast, eccentricity,
• below this threshold temporal changes are
  perceived as flicker or motion