Bayesian Perception

1
Bayesian Perception
2
General Idea
Ernst and Banks, Nature, 2002
3
General Idea

• Bayesian formulation

Conditional Independence assumption
4
General Idea
Generative model
w?
Ernst and Banks, Nature, 2002
5
General Idea
Probability
Width
6
General Idea
7
General Idea

• Mean and variance

8
General Idea
Probability
Width
v
t
9
General Idea

• Mean and variance

10
Optimal Variance

• Variance

Fisher information sums for independent signals
11
General Idea
Predicted by the Bayesian model
0.2
0.15
Threshold (STD)
0.1
0.05
0
0
67
133
200
Visual noise level ()
Note unimodal estimates may not be optimal but
the multimodal estimate is optimal
Ernst and Banks, Nature, 2002
12
Adaptive Cue Integration

• Note the reliability of the cue change on every
  trial
• This implies that the weights of the linear
  combination have to be changed on every trial!
• Or do they?

13
General Idea

• Perception is a statistical inference
• The brain stores knowledge about P(I,V) where I
  is the set of natural images, and V are the
  perceptual variables (color, motion, object
  identity)
• Given an image, the brain computes P(VI)

14
General Idea

• Decisions are made by collapsing the distribution
  onto a single value
• or

15
Key Ideas

• The nervous systems represents probability
  distributions. i.e., it represents the
  uncertainty inherent to all stimuli.
• The nervous system stores generative models, or
  forward models, of the world (P(IV)), and prior
  knowlege about the state of the world (P(V))
• Biological neural networks can perform complex
  statistical inferences.

16
Motion Perception
17
The Aperture Problem
18
The Aperture Problem
19
The Aperture Problem
20
The Aperture Problem
Vertical velocity (deg/s)
Horizontal velocity (deg/s)
21
The Aperture Problem
Vertical velocity (deg/s)
Horizontal velocity (deg/s)
22
The Aperture Problem
23
The Aperture Problem
Vertical velocity (deg/s)
Horizontal velocity (deg/s)
24
The Aperture Problem
Vertical velocity (deg/s)
Horizontal velocity (deg/s)
25
The Aperture Problem
Vertical velocity (deg/s)
Horizontal velocity (deg/s)
26
Standard Models of Motion Perception

• IOC interception of constraints
• VA Vector average
• Feature tracking

27
Standard Models of Motion Perception
IOC
VA
Vertical velocity (deg/s)
Horizontal velocity (deg/s)
28
Standard Models of Motion Perception
IOC
VA
Vertical velocity (deg/s)
Horizontal velocity (deg/s)
29
Standard Models of Motion Perception
IOC
VA
Vertical velocity (deg/s)
Horizontal velocity (deg/s)
30
Standard Models of Motion Perception
IOC
VA
Vertical velocity (deg/s)
Horizontal velocity (deg/s)
31
Standard Models of Motion Perception

• Problem perceived motion is close to either IOC
  or VA depending on stimulus duration,
  eccentricity, contrast and other factors.

32
Standard Models of Motion Perception

• Example Rhombus

Percept VA
Percept IOC
IOC
IOC
VA
VA
Vertical velocity (deg/s)
Vertical velocity (deg/s)
Horizontal velocity (deg/s)
Horizontal velocity (deg/s)
33
Moving Rhombus
34
Bayesian Model of Motion Perception

• Perceived motion correspond to the MAP estimate

35
Prior

• Human observers favor slow motions

36
Likelihood

• Weiss and Adelson

37
Likelihood
38
Likelihood
Binary maskPresumably, this is set by
segmentation cues
39
Posterior
40
Bayesian Model of Motion Perception

• Perceived motion corresponds to the MAP estimate

Only one free parameter
41
Likelihood
42
Motion through an Aperture

• Humans perceive the slowest motion.
• More generally we tend to perceive the most
  likely interpretation of an image

43
Motion through an Aperture
Likelihood
50
Vertical Velocity
0
-50
-50
0
50
ML
Horizontal Velocity
50
50
Vertical Velocity
Vertical Velocity
MAP
0
0
-50
-50
Prior
Posterior
-50
0
50
-50
0
50
Horizontal Velocity
Horizontal Velocity
44
Motion and Constrast

• Humans tend to underestimate velocity in low
  contrast situations

45
Motion and Contrast
Likelihood
50
Vertical Velocity
0
-50
High Contrast
-50
0
50
ML
Horizontal Velocity
50
50
Vertical Velocity
Vertical Velocity
MAP
0
0
-50
-50
Prior
Posterior
-50
0
50
-50
0
50
Horizontal Velocity
Horizontal Velocity
46
Motion and Contrast
Likelihood
50
Vertical Velocity
0
-50
Low Contrast
-50
0
50
ML
Horizontal Velocity
MAP
50
50
Vertical Velocity
Vertical Velocity
0
0
-50
-50
Prior
Posterior
-50
0
50
-50
0
50
Horizontal Velocity
Horizontal Velocity
47
Motion and Contrast

• Driving in the fog in low contrast situations,
  the prior dominates

48
Moving Rhombus
Likelihood
50
50
Vertical Velocity
Vertical Velocity
0
0
-50
-50
High Contrast
-50
0
50
-50
0
50
IOC
Horizontal Velocity
Horizontal Velocity
MAP
50
50
Vertical Velocity
Vertical Velocity
0
0
-50
-50
-50
0
50
-50
0
50
Prior
Posterior
Horizontal Velocity
Horizontal Velocity
49
Moving Rhombus
Likelihood
50
50
0
Vertical Velocity
0
Vertical Velocity
-50
-50
Low Contrast
-50
0
50
-50
0
50
IOC
Horizontal Velocity
Horizontal Velocity
50
50
MAP
Vertical Velocity
Vertical Velocity
0
0
-50
-50
-50
0
50
-50
0
50
Prior
Posterior
Horizontal Velocity
Horizontal Velocity
50
Moving Rhombus
51
Moving Rhombus

• Example Rhombus

Percept VA
Percept IOC
IOC
IOC
VA
VA
Vertical velocity (deg/s)
Vertical velocity (deg/s)
Horizontal velocity (deg/s)
Horizontal velocity (deg/s)
52
Barberpole Illusion
53
Plaid Motion Type I and II
54
Plaids and Contrast
Lower contrast
55
Plaids and Time

• Viewing time reduces uncertainty

56
Ellipses

• Fat vs narrow ellipses

57
Ellipses

• Fat vs narrow ellipses
• All motions agree

58
Ellipses
59
Ellipses

• Adding unambiguous motion

60
Ellipses

• Adding unambiguous motion

61
Other Prior

• Prior on direction of lightning

62
Generalization

• All computation are subject to uncertainty
  (ill-posed)
• This includes syntax processing, language
  acquisition etc.
• Solution compute with probability distributions

63
Binary Decision Making
Shadlen et al.
64
Race Model

• Standard theory some signal is accumulated (or
  integrated) to a bound. Also known as race
  models.
• The signal to be integrated could be the response
  of sensory neurons.

65
Bayesian Strategy

• The diffusion to bound model of Shadlen et al.

66
A Neural Integrator for Decisions?

• MT Sensory EvidenceMotion energy
• step

• LIP Decision FormationAccumulation of evidence
• ramp

67
Diffusion to bound model
68
Diffusion to bound model

• Proposed by Wald, 1947 and Turing (WW II,
  classified)
• Stone, 1960 then Laming, Link, Ratcliff, Smith,
  . . .

69
Diffusion to bound model
Criterion to answer Right
Accumulated evidencefor Rightwardandagainst
Leftward
Momentary evidencee.g.,?Spike rateMTRight
MTLeft
Criterion to answer Left
Seems arbitrary but why not?
Shadlen Gold (2004) Palmer et al (2005)
70
MT responses
60 40 20 0
Height scales with coherence
Firing rate
Right
Left
Direction (deg)
71
Diffusion to bound model

• Performance reaction time trade-off

72
Best fitting chronometric functionDiffusion to
bound
73
Predicted psychometric function Diffusion to
bound
74
Average LIP activity in RT motion task
choose Tin
choose Tout
Note the clear asymmetry
Roitman Shadlen, 2002 J. Neurosci.
75
Bayesian Strategy

• The Bayesian strategy in this case consists in
  computing the posterior distribution given all
  activity patterns from MT up to the current time,
  

76
Bayesian Strategy

• Race models and Bayesian approach

Temporal sum
Unless is related to

77
Bayesian Strategy

• Are neurons computing log likelihood?
• The difference of activity between two neurons
  with preferred directions 180 deg away is
  proportional to a log likelihood ratio.

78
Bayesian Strategy

• Log likelihood ratio

79
Bayesian Strategy

• Is the log likelihood ration proportional to
  ?

Coherence level
80
Bayesian Strategy

• Note that if you know , you
  still dont know the log likelihood ration unless
  youre given the coherence level.
• Therefore, the animal cant know its confidence
  level (the log likelihood ratio) unless it
  estimates C
• Another important point if we stop the race at a
  fixed level of we stop at
  different levels of log likelihood ratio
  depending on the coherence. This is why
  performance gets better when coherence increases,
  even though we always stop at the same activity
  threshold.

81
Decision Making

• Does that mean the animal does not know how much
  to trust its own decision?
• Does that mean the brain does not encode
  uncertainty or probability distribution?
• Seems unlikely

82

• To be continued