Bayesian modeling in the context of robust cue integration

1
Bayesian modeling in the context of robust cue
integration

• David C. Knill
• Center for Visual Science
• University of Rochester

2
The Bayesian approach as a framework for
psychophysics

• David C. Knill
• Center for Visual Science
• University of Rochester

3
Some properties of a useful psychophysical
framework

• Support building predictive models of perceptual
  performance.
• Support bridging statements between models and
  descriptions of behavior.
• Explain why perception / sensorimotor control
  works the way it does.
• Help guide psychophysical research
• Suggests new and interesting theoretical
  questions.
• Supports scaling down perceptual / sensorimotor
  problems to bring them into the lab.
• Scales up naturally

4
World model
5
World model
Noise
Sensory processing
Generative model
Sensory Features
Information
6
World model
Noise
Sensory processing
Generative model
Sensory Features
p(S I)
Bayesian Computations
7
World model
Noise
Sensory processing
Generative model
Estimate
Sensory Features
p(S I)
Bayesian Computations

Task model
8
World model
Noise
Sensory processing
Generative model
Estimate
Sensory Features
p(S I)
Bayesian Computations

Ideal Observer
Task model
9
World model
Noise
Sensory processing
Generative model
Estimate
Sensory Features
p(S I)
Bayesian Computations

Ideal Observer
Task model
10
Estimate
Sensory Features
Human Observer
11
Estimate
Sensory Features
p(S I)
Bayesian Computations

Task model
Rational Observer
12
World model
Generative model
Estimate
Sensory Features
p(S I)
Bayesian Computations

Task model
Rational Observer
13
Ideal observer models
Rational observer models
The domain of Bayesian models
Description of sensorimotor / perceptual behavior
14
Cue integrationEstimating slant from monocular
and binocular cues
15
(No Transcript)
16
Linear process model
Texture data (It)
Slant from texture
St
wt
Sst
Action / Decision

Stereo data (Is)
Ss
ws
Slant from stereo
17
Normative (ideal observer) model
Stereo likelihood
18
Normative (ideal observer) model
Texture likelihood
19
Normative (ideal observer) model
Joint likelihood
20
Humans weight sensory cues optimally

• Discrimination thresholds in single cue
  conditions predict weights measured in multi-cue
  experiments.
• Ernst and Banks, 2002 Knill and Saunders, 2003
  Alais and Burr (2004) etc., etc., etc.

21
Texture information
Binocular information
Least Reliable
Equally reliable
Most Reliable
22
What are cue weights?
23
What are cue weights?

• Summary descriptions of perceptual performance.

24
What are cue weights?

• Summary descriptions of perceptual performance.
• Summary descriptions of the information available
  for a task.

25
What are cue weights?

• Summary descriptions of perceptual performance.
• Summary descriptions of the information available
  for a task.
• Support logical links between behavior and
  rational / normative models of performance.

26
Robust non-linear cue integration
27
Robust non-linear cue integration

• Classical question
• How does the brain interpret multiple sensory
  cues when they have large conflicts

28
Robust non-linear cue integration

• Classical question
• How does the brain interpret multiple sensory
  cues when they have large conflicts
• For visual depth cues
• Re-conceptualize the problem
• what normally gives rise to what we call large
  cue conflicts?

29
Answer

• Most depth cues are informative because of
  statistical regularities in the world

30
Answer

• Most depth cues are informative because of
  statistical regularities in the world
• Examples
• Texture - isotropy, homogeneity
• Figure shape - isotropy, symmetry
• Motion - rigidity

31
Answer

• Most depth cues are informative because of
  statistical regularities in the world
• Examples
• Texture - isotropy, homogeneity
• Figure shape - isotropy, symmetry
• Motion - rigidity
• Constraints dont always apply
• True prior model mixture of constraints

32
Answer

• Most depth cues are informative because of
  statistical regularities in the world
• Examples
• Texture - isotropy, homogeneity
• Figure shape - isotropy, symmetry
• Motion - rigidity
• Constraints dont always apply
• True prior model mixture of constraints
• Large conflicts arise when strong constraint
  does not hold.

33
Compression cue slant suggested by circle
interpretation
34
Likelihood function -
p(circle)

p(ellipse)
35
Figure shape cue
36
Disparity cue
37
Combined cues
38
(No Transcript)
39
(No Transcript)
40
(No Transcript)
41
(No Transcript)
42
(No Transcript)
43
(No Transcript)
44
(No Transcript)
45
(No Transcript)
46
Predictions of mixture model
All circles
Circles narrow range of ellipses
Circles broad range of ellipses
47
(No Transcript)
48
Stereoscopic slant 35o
49
Model fits
Stereoscopic slant 35o
50
Model fits
Stereoscopic slant 55o
51
(No Transcript)
52
Conclusions

• Real cue integration problems are highly
  non-linear
• The linear-gaussian Bayesian model scales up
  easily - e.g. add a mixture component to the
  prior on shape.
• The linear-gaussian process model does not scale
  up.
• Bayesian formulation suggests new questions
• How does rational observer model compare to a
  normative model.
• How do observers adapt their prior models across
  different environments?

53
Bayesian learning model
Disparities
Slant
Estimate slant and shape
Figure Shape
Retinal shape
Prior distribution of figure shapes
54
Behavior of the learning model
55
Results of matching experiment
56
Thank you