Leonardo Chelazzi Verona

1
Interacting Competitive Selection Across Levels
of Representation
Leonardo Chelazzi (Verona) Robert Desimone
(NIMH) Wayne Khoe (UCSD) Steve Hillyard (UCSD)
Jude Mitchell (Salk) Tania Pasternak
(Rochester) Gene Stoner (Salk) John Reynolds
(Salk)
2
Faculty retreat Oct 14-16u
3
Talk Outline Vision as a limited capacity
information processing system
4
Talk Outline Vision as a limited capacity
information processing system Extrastriate
visual cortex plays a key role in determining
which information passes through this limited
capacity system
5
Talk Outline Vision as a limited capacity
information processing system Extrastriate
visual cortex plays a key role in determining
which information passes through this limited
capacity system Competitive mechanisms in
extrastriate cortex select salient or
behaviorally relevant stimuli
6
Talk Outline Vision as a limited capacity
information processing system Extrastriate
visual cortex plays a key role in determining
which information passes through this limited
capacity system Competitive mechanisms in
extrastriate cortex select salient or
behaviorally relevant stimuli Feedback from
areas outside visual cortices, including the
occulomotor system, bias this competitive
selection
7
Talk Outline Vision as a limited capacity
information processing system Extrastriate
visual cortex plays a key role in determining
which information passes through this limited
capacity system Competitive mechanisms in
extrastriate cortex select salient or
behaviorally relevant stimuli Feedback from
areas outside visual cortices, including the
occulomotor system, bias this competitive
selection
8
(No Transcript)
9
(No Transcript)
10
(No Transcript)
11
(No Transcript)
12
(No Transcript)
13
Talk Outline Vision as a limited capacity
information processing system Extrastriate
visual cortex plays a key role in determining
which information passes through this limited
capacity system Competitive mechanisms in
extrastriate cortex select salient or
behaviorally relevant stimuli Feedback from
areas outside visual cortices, including the
occulomotor system, bias this competitive
selection
14
(No Transcript)
15
Schiller, Visual Neuroscience 1993 Mild
deficits in an array of discrimination tasks
including form vision, flicker perception,
wavelength discrimination. Major deficit in
selecting a less prominent stimulus from among
more prominent distracters.
16
De Weerd et al, Nature Neuroscience 1999 Mild
deficits orientation discrimination. Major
deficit in selecting a low contrast stimulus from
among high contrast distracters.
17
De Weerd et al, Nature Neuroscience 1999
18
Distracters cannot be ignored
De Weerd et al, Nature Neuroscience 1999
19
Talk Outline Vision as a limited capacity
information processing system Extrastriate
visual cortex plays a key role in determining
which information passes through this limited
capacity system Competitive mechanisms in
extrastriate cortex select salient or
behaviorally relevant stimuli Feedback from
areas outside visual cortices, including the
occulomotor system, bias this competitive
selection
20
Spatial Attention Task
Test Trials
Instruction Trials
Fixation Point
CueBox
Non Target
Distractor
Receptive Field
Target
21
Preferred, Att Away
V4
80
60
Spikes per second
40
20
0
100
200
300
0
ms. after stimulus onset
Poor, Att Away
22
Pair, Att Preferred
Preferred, Att Away
Attention
V4
80
60
Spikes per second
40
20
0
100
200
300
0
ms. after stimulus onset
Poor, Att Away
23
Pair, Att Preferred
Preferred, Att Away
Attention
V4
80
60
Pair, Att Preferred
Spikes per second
40
Attention
20
0
100
200
300
0
ms. after stimulus onset
Poor, Att Away
24
What sort of cortical circuit would respond this
way?
25
Individual responses
Hypothetical Cortical Circuit
1
Recording Electrode
0.8
0.6
0.4
0.2
0
log(contrast)
Orientation Tuning vs. contrast
1
0.8
0.6
0.4
0.2
Normalization Models Sperling and Sondhi,
1968 Grossberg, 1973 Albrecht and Geisler,
1991 Carandini and Heeger, 1994 Reynolds,
Chelazzi and Desimone, J. Neurosci. 1999
0
-50
0
50
26

• Assumptions
• Neuronal response depends on the summed
  excitatory and inhibitory inputs to stimuli in
  the RF.

Suppression By Poor Stimulus
Response (spikes per second)
Time from stimulus onset
27

• Assumptions
• Neuronal response depends on the summed
  excitatory and inhibitory inputs to stimuli in
  the RF.

Suppression By Poor Stimulus
Response (spikes per second)
Time from stimulus onset
28

• Assumptions
• Neuronal response depends on the summed
  excitatory and inhibitory inputs to stimuli in
  the RF.

Suppression By Poor Stimulus
Response (spikes per second)
Time from stimulus onset
29

• Assumptions
• Neuronal response depends on the summed
  excitatory and inhibitory inputs to stimuli in
  the RF.

Suppression By Poor Stimulus
Response (spikes per second)

• Predictions
• Adding a poor stimulus to a preferred stimulus
  should suppress the response

Time from stimulus onset
30

• Assumptions
• Neuronal response depends on the summed
  excitatory and inhibitory inputs to stimuli in
  the RF.
• When the animal discriminates a stimulus, this
  causes a proportional increase in the strength of
  both excitatory and inhibitory inputs from cells
  that are driven by that stimulus.

Suppression By Poor Stimulus
Response (spikes per second)

• Predictions
• Adding a poor stimulus to a preferred stimulus
  should suppress the response

Time from stimulus onset
31

• Assumptions
• Neuronal response depends on the summed
  excitatory and inhibitory inputs to stimuli in
  the RF.
• When the animal discriminates a stimulus, this
  causes a proportional increase in the strength of
  both excitatory and inhibitory inputs from cells
  that are driven by that stimulus.

Suppression By Poor Stimulus, Magnified
Response (spikes per second)

• Predictions
• Adding a poor stimulus to a preferred stimulus
  should suppress the response
• Attention to the poor stimulus should magnify
  this suppression.

Time from stimulus onset
32
V2
Preferred Att Away
70
60
50
40
Spikes per second
30
20
10
SEM
200
0
0
50
100
150
200
ms. after stimulus onset
Reynolds, Chelazzi and Desimone, 1999, J.
Neurosci.
33
V2
Preferred Att Away
70
Poor, Att Away
60
50
40
Spikes per second
30
20
10
SEM
200
0
0
50
100
150
200
ms. after stimulus onset
Reynolds, Chelazzi and Desimone, 1999, J.
Neurosci.
34
V2
Pair, Att Away
Preferred Att Away
70
Poor, Att Away
60
50
40
Spikes per second
30
20
10
SEM
200
0
0
50
100
150
200
ms. after stimulus onset
Reynolds, Chelazzi and Desimone, 1999, J.
Neurosci.
35
V2
Pair, Att Away
Preferred Att Away
70
Poor, Att Away
60
50
40
Spikes per second
Pair, Att Poor
30
Attention
20
10
SEM
200
0
0
50
100
150
200
ms. after stimulus onset
Reynolds, Chelazzi and Desimone, 1999, J.
Neurosci.
36

• Assumptions
• Neuronal response depends on the summed
  excitatory and inhibitory inputs to stimuli in
  the RF.
• When the animal discriminates a stimulus, this
  causes a proportional increase in the strength of
  both excitatory and inhibitory inputs from cells
  that are driven by that stimulus.

Suppression By Poor Stimulus
Response (spikes per second)

• Predictions
• Adding a poor stimulus to a preferred stimulus
  should suppress the response
• Attention to the poor stimulus should magnify
  this suppression.

Time from stimulus onset
3) Attention to the preferred stimulus should
diminish suppression.
37

• Assumptions
• Neuronal response depends on the summed
  excitatory and inhibitory inputs to stimuli in
  the RF.
• When the animal discriminates a stimulus, this
  causes a proportional increase in the strength of
  both excitatory and inhibitory inputs from cells
  that are driven by that stimulus.

Suppression By Poor Stimulus, Magnified
Response (spikes per second)

• Predictions
• Adding a poor stimulus to a preferred stimulus
  should suppress the response
• Attention to the poor stimulus should magnify
  this suppression.

Time from stimulus onset
3) Attention to the preferred stimulus should
diminish suppression.
38
Preferred, Att Away
V4
80
60
Spikes per second
40
20
0
100
200
300
0
ms. after stimulus onset
Poor, Att Away
39
Preferred, Att Away
V4
80
Pair, Att Preferred
60
Spikes per second
Attention
40
20
0
100
200
300
0
ms. after stimulus onset
Poor, Att Away
40
Preferred, Att Away
V4
80
Pair, Att Preferred
60
Spikes per second
Attention
40
20
0
100
200
300
0
ms. after stimulus onset
Poor, Att Away
41
Prediction 4 when the animal discriminates a
stimulus this will result in a leftward shift in
the contrast response function.
Recording Electrode
Normalized Response
-10
-5
0
5
Saturation Contrast
Threshold Contrast
Log Contrast
42
Test Trials
Instruction Trials
Fixation Point
Non Target
Distractor
CueBox
Receptive Field
Target
Reynolds, Pasternak and Desimone, Neuron, 2000
43
(No Transcript)
44
Average Responses, all Neurons (N84)
50
50
50
50
50

















Attend RF
40
40
40
40
40
30
30
30
30
30
Attend away
Spikes per second
attend
20
20
20
20
20
10
10
10
10
10
ignore
0
0
0
0
0
0
200
400
0
200
400
0
200
400
0
200
400
0
200
400
Sub-Threshold Contrast
Saturation Contrast
Time from stimulus onset (ms)
45
Attention Increases Neuronal Sensitivity


attend
40
40


N39
30
30

Change in Response
Firing Rate (spikes/sec)
20
20
ignore
10
10
0
0
46
Neurometric Function
1
Threshold Attend RF
ROC Value
Threshold Attend Away
0.5
1
10
100
Contrast
Distribution of Threshold Shifts
18
12
Number of Neurons
6
0
-.6
0
.6
.9
.3
-.3
-.9
Change in Contrast Threshold Log (Threshold
Attend RF / Threshold Attend Away)
47
(No Transcript)
48
Responses to individual stimuli
1
0.8
0.6
0.4
0.2
Prediction 5 attention should cause a gain
multiplication in the tuning curve.
0
log(contrast)
Orientation Tuning with and without attention
1
0.8
0.6
0.4
0.2
0
-50
0
50
Feature Value (e.g., orientation, direction of
motion)
49
(No Transcript)
50
Spatial attention modulates gain of tuning curve
McAdams and Maunsell, 1999 Treue and
Martinez-Trujillo, Nature, 1999
51
Suppression By Poor Stimulus, Magnified
Prediction 6 Physically increasing the strength
of the poor stimulus should magnify suppression,
as attention does.
Response (spikes per second)
(5)
Time from stimulus onset
Increased luminance contrast
52
Stimuli
Preferred
Stimulus at
Fixed Contrast
Poor
Stimulus
5 Contrasts
Resulting
5 Pairs
Poor Probe Contrast Increasing
53
5
10
15
Reference Stimulus
20
5
10
15
Probe Stimulus
20
5
10
15
20
Time from Stimulus Onset (ms.)
Reynolds and Desimone, 2003, Neuron
54
5
10
15
Reference Stimulus
20
5
10
15
Probe Stimulus
20
5
10
15
20
Time from Stimulus Onset (ms.)
Reynolds and Desimone, 2003, Neuron
55
5
10
15
Reference Stimulus
20
5
10
15
Probe Stimulus
20
5
10
15
20
Time from Stimulus Onset (ms.)
Reynolds and Desimone, 2003, Neuron
56
Reynolds and Desimone, 2003, Neuron
57
Suppression by Probe, Response to Probe (N39)
5
0
-5
Effect of Adding Probe s/sec
-10
-15
-20
0
200
400
40
30
Response to Probe Alone s/sec
20
10
0
200
400
Time from stimulus onset (ms.)
58
Suppression by Probe, Response to Probe (N39)
5
0
-5
Effect of Adding Probe s/sec
-10
-15
-20
0
200
400
40
30
Response to Probe Alone s/sec
20
10
0
200
400
Time from stimulus onset (ms.)
59
Suppression by Probe, Response to Probe (N39)
5
0
-5
Effect of Adding Probe s/sec
-10
-15
-20
0
200
400
40
30
Response to Probe Alone s/sec
20
10
0
200
400
Time from stimulus onset (ms.)
60
Suppression by Probe, Response to Probe (N39)
5
0
-5
Effect of Adding Probe s/sec
-10
-15
-20
0
200
400
40
30
Response to Probe Alone s/sec
20
10
0
200
400
Time from stimulus onset (ms.)
61
Suppression by Probe, Response to Probe (N39)
5
0
-5
Effect of Adding Probe s/sec
-10
-15
-20
0
200
400
40
30
Response to Probe Alone s/sec
20
10
0
200
400
Time from stimulus onset (ms.)
62
Suppression by Probe, Response to Probe (N39)
Contrast of Probe
Highest
40
Lowest
30
Response to Probe Alone s/sec
20
10
0
200
400
Time from stimulus onset (ms.)
c.f. Carandini, Heeger and Movshon, J.Neurosci,
1997
63
Synchrony A possible mechanism by which
attention increases stimulus strength
Theory Crick and Koch, 1990 Niebur and Koch,
1990 Salinas and Sejnowski, 2000 Data Makeig and
Jung, 1996 Alonso Usrey and Ried, 1996 Azouz and
Gray, 2000 Steinmetz et al., 2000
64
Spikes
Voltage
Time
Local Field Potentials
Voltage
Time
65
Computing Spike-triggered-average Local field
potential (STA)
Spikes
Local Field Potentials
66
Computing Spike-triggered-average Local field
potential (STA)
0
Voltage
Time relative to spike
Voltage
67
Computing Spike-triggered-average Local field
potential (STA)
0
Voltage
Time relative to spike
0
Voltage
68
Computing Spike-triggered-average Local field
potential (STA)
0
Voltage
Time relative to spike
0
Voltage
69
Computing Spike-triggered-average Local field
potential (STA)
0
Voltage
Time relative to spike
0
Voltage
Average across 1000s of spikes
(STA)
0
70
(No Transcript)
71
(No Transcript)
72
(No Transcript)
73
Prestimulus Period
1

100
te spikes/s
Firing ra
0
0
1
2
3
4
-1
Time f
rom stimulu
s onset s

2
0.08
V
m
1
1
V

m

power
0
0
STA
STA
-1
-1
0
0
100
-100
0
40
80
0
100
-100
Time
shift ms
Frequency Hz
Time
shift ms
Attend outside RF
Attend inside RF
74
(No Transcript)
75
Changes in synchrony with attention
1

100
te spikes/s
Firing ra
0
0
1
2
3
4
-1
Time f
rom stimulu
s onset s

2
V
0.025
1
1
m
V

m

0
0
power
STA
-1
-1
STA
0
0
100
-100
0
100
-100
0
40
80
Frequency Hz
Time
shift ms
Time
shift ms
Attend outside RF
Attend inside RF
76
(No Transcript)
77
Changes in synchrony are spatially selective
1

te spikes/s
200
Firing ra
0
0
1
2
3
4
-1
Time f
rom stimulu
s onset s

2
V
m
V
0.01

0
0
m

power
STA
-1
-1
STA
0
0
40
80
0
100
-100
0
100
-100
Frequency Hz
Time
shift ms
Time
shift ms
Attend outside RF
Attend inside RF
78
Talk Outline Vision as a limited capacity
information processing system Extrastriate
visual cortex plays a key role in determining
which information passes through this limited
capacity system Competitive mechanisms in
extrastriate cortex select salient or
behaviorally relevant stimuli Feedback from
areas outside visual cortices, including the
occulomotor system, bias this competitive
selection
79
(No Transcript)
80
(No Transcript)
81
Summary The visual system is limited in
capacity Lesion studies demonstrate the
importance of extrastriate visual cortex in
determining which information passes through this
limited capacity system Single-unit recording
studies support a model in which competitive
mechanisms in extrastriate cortex select salient
or behaviorally relevant stimuli Feedback from
areas outside visual cortices, including the
occulomotor system, bias this competitive
selection
82
(No Transcript)