A Review on Saliency Computation Methods

1
A Review on Saliency Computation Methods

• Papers reviewed
• A Model of Saliency-based Visual Attention for
  Rapid Scene analysis(Itti,Koch,Niebur).
• A Saliency-Based Search Mechanism for Overt and
  Covert Shifts of Visual Attention(Itti,Koch)

2
Introduction

• Primates. Object identification involves either
  rapid, saccadic eye movements to bring the fovea
  onto the object.
• Only a small part of the information registered
  by the visual system at any given time reaches
  levels of processing that directly influence
  behaviour.
• Preattentive selection gives more importance to
  some image regions(salient regions)

3
Model

• Bottom-up control of attention. Localization of
  the stimuli.(Where, not What)
• Focus of Attention(FOA).Region which the system
  is looking.
• Inhibition of Return(IOR). The FOA at time t is
  inhibited during constant time. Well demostrated
  for covert attentional shifts in humans.
• Static color images. Nine scales are created
  using Gaussian pyramids(progressively low-pass
  filter and subsample the image), from 11 to
  1256.

4
Some definitions

• Center-surround difference. Given two scaled
  images I(i),I(j), is obtained by interpolation
  to the finer scale and point-by-point
  subtraction. Denoted by
• Across scale addition. Reduction of a map to
  scale 4 and point-by-point addition. Denoted by

5
Early Visual Features

• Intensity contrast. Detected in mammals by
  neurons sensitive to either dark centers-bright
  surrounds, bright centers-dark surrounds.

6
Early Visual Features(2)

• Center of the receptive field, neurons
  excited by one color and inhibited by another. In
  human primary visual cortex are red/green,
  green/red, blue/yellow, yellow/blue.

7
Early Visual Features(3)

• Orientation-selective neurons in primary visual
  cortex

8
Normalization operator

• Normalize the values in the map to a fixed
  range0...M.
• Find the location of the maps global maximum M
  and computing the average m of all its other
  local maxima.
• Globally multiplying the map by (M-m)2
• This factor replicates cortical lateral
  inhibition mechanisms, in which neighborging
  similar features inhibit each other.

9
Saliency Map(Version 1)

• The motivation of three diferent values is that
  similar values compete strongly for saliency,
  while different modalities contribute indepently
  to the saliency map

10
Saliency Map(Version 2)

• After applying the normalization factor, this map
  modification is done 10 times.

The above map modification is applied to these
three maps. Finally,
11
Version 1
Version 2
12
Comments

• In the version 1, it fails to reliably detect
  pop-outs in search arrays

13
Comments(2)

• There is no top-down process.
• Does not include any measure of saliency based on
  temporal stimulus onset or dissapearance or on
  motion. Neither grouping of stimuli.
• Very robust to the addition of noise.
• There is no any recurrent mechanism(cannot
  reproduce phenomena like contour completion and
  closure, important for certain types of human
  pop-out).

14
Questions

• Does saliency maps involve different
  behaviours/different visual-motor patterns?
• If saliency is encoded across multiple maps, how
  competition can act across these maps to ensure
  that only a single location is chosen?
• Saliency is encoded in one/many maps?
• In which way the top-down and bottom-up processes
  are related?
• How many features should be in mind to obtain all
  relevant saliency map(s)?

15
Related papers(BMCV 2002)

• Biologically inspired saliency map model for
  bottom-up visual attention.(Park, Shin, Lee) Here
  there is an implementation of the saliency model
  presented, but with other features(edges,
  simmetry and color differences).
• A goal oriented attention guidance
  model.(Navalpakkam, Itti). They uses the saliency
  map just presented, with a topographic
  task-relevance map. Basically, the saliency map
  filters the image to be used by the
  task-relevance map.