A biologically useful memory

1
A biologically useful memory mechanism for the
rapid deployment of visual attention
Ken Nakayama
2
seeing visibility X attention
R

3
Metaphors for vision

• Camera Its like a picture
• Hand its more active

Attention is the hand
How is it controlled?
4
Backgroundsome robust examples of attention

• Change blindness
• Inattentional blindness (even at fovea)
• Attentive tracking hand and fingers

5
Rensink flicker experiment

• method alternate two pictures
• Ask subjects to identify changes
• If we were aware of everything in picture, should
  be easy
• raise your hand when you see the change dont
  tell others

At Nissan CBR
6
Rensink et al., Simons and Levin Change
Blindness
7
(No Transcript)
8
summary of Rensink flicker experiment
contrary to our phenomenological experience we
are not aware of everything in our visual world
large changes can escape our notice
Conclusion seeing requires attention
9
can change size and position approx 4-6 times/sec
10
Other functions of attention

• Guidance of motor behavior
• Foraging for food

11
(No Transcript)
12
(No Transcript)
13
The intensity of predation depends . . . on
the use of specific searching images. This
implies that the birds perform a highly
selective sieving operation on the visual
stimuli that reach their retina . . . birds can
only use a limited number of different search
images at the same time. L. Tinbergen(1960)
14
Marian Dawkins (1971) Shifts of attention in
chicks during feeding
15
Search image widespread?

• Chicks (Dawkins, 1971)
• Pigeons (Reid and Shettleworth, 1992))
• Blue Jays (Bond)
• Bumble Bees (1992)
• Butterflies (Stanton, 1984)

Is it a mental image or could it be something
else?
16
ClaimThere exists a primitive distributed
memory system (seen in our human experiments)
that could account for shifts of attention
attributed to search images
17
It is part of a fast transient attentional system

• (Nakayama and Mackeben, 1989
• Vision Research)

18
(No Transcript)
19
Isolation of the transient component
Keep location constant
20
Is the transient component due to its activation
by a sensory transient ?

• NO
• such sensory transients not necessary

analogy to action potential ?
21
Deploy transient attention without a
local sensory transient
22
decoy cueing
23
Transient attention is very fast, rises to peak
within 100 msec.
Is it fast AND flexible

• Can transient attention learn to go quickly to
  the appropriate position on a larger object ?

Kristanjansson, Mackeben Nakayama, 2001
24
(No Transcript)
25
Attention can be effectively deployed toa
location within anobject
fixed
variable
26
Is the learning a property of sustained or
transient attention ?
27
Keep target position within the cue constant
28
Learning

• How fast does it occur ?
• How stable is it ?
• Method use quasi random streaks of cue target
  regularities

29
Build-up of learning
time
1
2
5
sequence of cues
30
color ?
streak
31
local shape ?
32
Learning (summary so far)

• Is very rapid and is temporary
• Can be linked position within an object
• Can linked to a color within an object
• Can linked to a local shape within an object

Are there things attention cant learn ?
33
can it learn a 2nd order relation?


shape and position
color and position
34
AK
2 subjects
AMH
random
consistent
35
Part of a fast mechanism of Attentional
deployment , reaches peak with 100 msec
36
Different paradigm to study the same process
n
position and color of the target can change
n-1
Identify shape of the odd colored target
Maljkovic and Nakayama
37
.
.
n
1
2
repeat color
38
Repeat target position
.
.
1
2
n
39
Attention can learn colors, shapes and locations
Approach manipulate target color uncertainty
40
?
41
(No Transcript)
42
make expectancy very explicit maximizing the
possible use of search images, pitting it
against repetition
43

Trial

sequence

predicted outcomes
repetition
reaction time
Search image (expectancy)
2
1
order in sequence
44
(No Transcript)
45
Learning is passive, mechanistic,
piecemeal(color, position)

• Its not expectancy,
• Its not a search image
• Its not under conscious control

46
Its not just a linkage to the previous targets
but active inhibition to non-targets
47
(No Transcript)
48
Fine grain temporal analysis of the learning
mechanism
What is the influence of a single trial in the
past ?
2nd order reverse correlation
49
current trial
What is influence of a single trial in the past ?
50
(No Transcript)
51
(No Transcript)
52
Linkage is not confined to color

• Probably any salient feature
• (spatial frequency, for example), will do

53
(No Transcript)
54
Learning restricted to what attracts
attention(features, positions)

• Not the fine details that attention allows one to
  process

55
(No Transcript)
56
Is the build up and decay over time or over
events?
57
Independence of features/location
58
Effect of target and distractor position in the
past
59
influence of trial n-1
60
Independence of color and location learning
61
hypothesized properties of the memory system
graded summates linearity (superposition) has
independent components (features and locations)
62
(No Transcript)
63
What is learned ?What is the reinforcer ?
Simple identifiers of places where attention just
wentand didnt go ?
64
Generalizations beyond the measure of attention

• Speeds eye movements (human and monkey)
• Speeds motor behavior (manual pointing)

65
Saccadic eye movements(in human and in monkey)
66
Human eye movements
Task make a saccade to the odd colored target
67
Cumulative effects of learning
saccadic latency
position in same color sequence
McPeek, Maljkovic Nakayama
68
Learning of speeded saccades in monkey
Saccadic latency
Position in same color sequence
Robert McPeek and Ed Keller Smith Kettlewell
69
Learning generalizes to manual pointing
Measure RT with Touch sensitive screen
(Song Nakayama, 2003)
70
Learning generalizes to manual pointing
Measure RT with Touch sensitive screen
(Song Nakayama, 2003)
71
Touching target with finger
72
Implications for foraging

• Dont need a cognitive concept like a search
  image
• Low level temporary, passive, graded connection
  strengths (plus and minus) may be sufficient

73
Marian Dawkins (1971) Shifts of attention in
chicks during feeding
74
Anatomical locale/mechanism?

• Object centered
• not retinotopic cortex
• Independent features
• simple 2 layered network

75
Relationship to other learning systems

• Is it a completely specialized sub-system for
  attention ?
• OR
• Is it the germ for short term memory more
  generally ?

76
A biologically useful and conserved memory
mechanism for the rapid deployment of visual
attention (and other possible functions
Ken Nakayama Manfred Mackeben Vera
Maljkovic Robert McPeek Arni Kristjansson Joo-Hyun
Song