The Selective Tuning Model of Visual Attention

1
Eye Movements andWorking Memory Marc
Pomplun Department of Computer
Science University of Massachusetts at
Boston E-mail marc_at_cs.umb.edu Homepage
http//www.cs.umb.edu/marc/
2
Eye Movements andWorking Memory

• Overview
• Image Processing Convolution Filters
• Iconic Memory Representations for Visual Search
• Working Memory Use in a Natural Task
• The Working Memory - Eye Movement Tradeoff

3
Convolution Filters

• Grayscale Image

Averaging Filter
4
Image Processing

• Original Image

Filtered Image
5
value 1?1/9 6?1/9 3?1/9 2?1/9 11?1/9
3?1/9 5?1/9 10?1/9 6?1/9 47/9
5.222
5
Image Processing

• Original Image

Filtered Image
5
7
value 6?1/9 3?1/9 2?1/9 11?1/9 3?1/9
10?1/9 10?1/9 6?1/9 9?1/9 60/9
6.667
6
Image Processing

• Original Image

Filtered Image
5
7
5
5
5
6
5
6
4
Now you can see the averaging (smoothing) effect
of the 3?3 filter that we applied.
7
Gaussian Filters

• implement decreasing influence by more distant
  pixels

8
Gaussian Filters

• Effect of Gaussian smoothing

9
Sobel Filters

• Sobel filters are an example for edge detection
  filters.
• Two small convolution filters are used
  successively

10
Sobel Filters

• Sobel filters yield two interesting pieces of
  information
• The magnitude of the gradient (local change in
  brightness)

• The angle of the gradient (tells us about the
  orientation of an edge)

11
Sobel Filters
Original image (left) and result of calculating
the magnitude of the brightness gradient with a
Sobel filter (right)
12
Rao, Zelinsky, Hayhoe Ballard (2002)Eye
Movements in Iconic Visual Search

• Question How do people represent items in their
  memory for efficient visual search?
• Idea Iconic (appearance-based) multiscale
  representation
• Such representations were modeled using
  spatiochromatic convolution filters of different
  scales and orientations.

13
Rao, Zelinsky, Hayhoe Ballard

• Convolution filters used for the model.

14
Rao, Zelinsky, Hayhoe Ballard

• According to the model, iconic visual search
  proceeds as follows
• The first saccade is aimed at the point in the
  visual scene whose low-frequency features
  have the best match with the low-frequency
  features of the memorized object.
• For the programming of the following
  saccades, higher and higher frequencies are
  included, until the target is found.

15
Rao, Zelinsky, Hayhoe Ballard

• Coarse-to-fine scanning mechanism

16
Rao, Zelinsky, Hayhoe Ballard

• Conclusion Good correspondence between modeled
  and empirical scanpaths

17
Ballard, Hayhoe Pelz (1995)Memory
Representations in Natural Tasks

• Task Copy a pattern of colored blocks

18
Ballard, Hayhoe Pelz

• Possible strategies for completing the block
  copying task. Participants performed the
  following operations(M)odel inspection,
  (P)ickup, and (D)ropoff.

19
Ballard, Hayhoe Pelz

• Typical hand and gaze trajectories for a single
  copying step

20
Ballard, Hayhoe Pelz

• Empirical frequency of individual strategies in
  the block copying task

21
Ballard, Hayhoe Pelz

• Conclusions
• In the block copying task, working memory is only
  sparsely used.
• Instead, subjects prefer to make additional eye
  movements.
• Because eye movements are inexpensive, subjects
  use the visual scene as an external memory
  rather than building an internal representation
  of it.

22
Eye Movement - Working MemoryTradeoff (Inamdar
Pomplun, 2003)

• Based on the previous study by Ballard et al, it
  seems that using working memory is clearly more
  expensive than performing eye movements.
• So maybe a cost model is an adequate way of
  describing and predicting behavior in visual
  tasks.

23
Inamdar Pomplun

• The basic idea is that the visual system
  (including the cognitive mechanisms that are
  required for performing the task) optimizes
  visual behavior, i.e. minimizes its effort
  (cost).
• Is there such a tradeoff between the use of
  working memory and eye movements?
• If so, what exactly is minimized? Can this be
  quantified?

24
Inamdar Pomplun

• Let subjects perform a visual task that requires
  eye movements and extensive use of visual working
  memory.
• Vary the cost of eye movements.
• Hypothesis If the assumed tradeoff between eye
  movements and working memory exists, costlier eye
  movements should lead to increased use of working
  memory.

25
Stimuli in Experiment 1

• Subjects were presented with two columns of
  simple geometrical objects in three different
  colors and three different shapes.
• The columns were identical except for one object
  that differed in either its color or its shape
  (in target-present trials).
• Subjects had to indicate whether such a target
  was present or not.
• The objects in the non-attended hemifield were
  always masked.
• The cost of eye movements was varied by changing
  the distance between the two columns.

26
Stimuli in Experiment 1
27
Stimuli in Experiment 1
28
Stimuli in Experiment 1
29
Stimuli in Experiment 1
30
Eye Movements in Experiment 1
31
Eye Movements in Experiment 1
32
Results of Experiment 1
33
Results of Experiment 1
34
Results of Experiment 1
35
Experiment 2

• What happens if the capacity limit of visual
  working memory is reached?
• By just varying the distance between columns, the
  cost of eye movements cannot be dramatically
  increased.
• Idea Artificially increase the cost of eye
  movements in the present paradigm by delaying the
  unmasking of objects after any gaze switch
  between hemifields.

36
Stimuli in Experiment 2

• We used the same stimuli as in Experiment 1, but
  only those for the medium-distance condition.
• Three visibility delays were used 0ms, 500ms,
  and 1000ms.
• During the delays, objects in both hemifields
  were masked.

37
Results of Experiment 2
38
Results of Experiment 2
39
Results of Experiment 2
40
Conclusions

• There clearly is a cost-minimizing behavior with
  regard to eye movements and working memory.
• However, the current data does not allow to build
  a quantitative model of this phenomenon.
• It seems that people slightly overestimate their
  working memory capacity when they are forced to
  heavily increase their working memory load.