The binding problem

1
The binding problem

• Background and approaches

2
Overview

• The visual system and some of its properties
• Theory of feature integration
• Binding without consciousness
• Synchronicity and consciousness
• Final doubts

3
The Visual System
4
Cortical processing

• Retinal (visual) input is processed by a
  hierarchy of visual modules
• Modularity in the visual system
• But to what extent?
• Functional specialization
• color, shape, motion,
• Feature maps in the visual system
• For each visual point ( retinal receptor)
• detector-cells in all orientations
• Retinotopic organization
• V1 neurons are spatially organized in a way that
  reflects the spatial organization of the retina

5
Retinotopic maps in the visual system

• The visual system has many retinotopically-organi
  zed feature maps

• Two questions
• Why are there so many visual areas?
• Why are features detected in special maps?

6
Binding Feature integration theory

• Squarely in psychology
• Weak ai variant
• Model of cognitive system
• Deals with the functional mind
• For time being leaving out the phenomenal mind
• A theory by Anne Treisman
• Connects psychological results on feature search
  behavior with biological data
• Feature Integration Theory (FIT)
• Master Map of Locations theory

7
Feature Integration Theory
Odd-one-out detection
8
Texture segregation
9
Assumptions

• Assummption 1
• Features. Visual scenes are decomposed into
  elementary features such as
• oriented edges
• colors
• shapes
• sizes
• Explains pop-out phenomena (i.e., the fast
  detection of the odd-one-out)
• Assumption 2
• Objects
• Features are bound to form objects (which were
  decomposed in the feature maps)
• The process of binding requires attention
• Explains the slow detection of feature
  conjunctions

10
Two stages

• Stage 1 feature extraction from image
• preattentive
• Stage 2 identification of objects (or regions)
• attentive
• Therefore,
• Differences in line orientations allow for
  pre-attentive segregation or grouping
• Differences in arrangements of lines require
  attentive segregation

11
RT curves for parallel / serial search
12
Location is special in FIT

• In the pre-attentive stage, features are detected
  in parallel, but not their locations
• A special feature map for locations (master map
  of locations) links the appropriate features
  with their location
• So, location is only known in the attentive stage
• Makes for illusory conjunctions

13
Instruction

• You will see a figure. There are two large
  numbers on the left and right of this figure.
  Determine if they are both odd

14

15
(No Transcript)
16
(No Transcript)
17
Test

• Did the figure contain
• The letters R, P, or Q?
• Did you see a vertical yellow bar?
• Did you see a horizontal green bar?
• If you saw an R, a Q, or or a horizontal
  green bar you have made an illusory conjunction
  error

18
The figure once more

• R, P, Q, vertical yellow bar, horizontal green
  bar?

19
Conclusions on FIT

• Features are processed in parallel
• Conjunctions have to be glued together (i.e.,
  binding) and are therefore processed in a serial
  fashion (attentional spotlight)
• Return to the 2 questions at the beginning
• Why are there so many visual areas?
• to detect elementary features
• Why are features detected in special maps?
• odd-one-out detection using lateral inhibition

20
The Binding problem

• Part 1

21
What is the binding problem?
Shape map
Colour map
Motion map
retina
Homunculus
22
An example of visual binding
23
An example of visual binding
Kanizsa triangle withillusory contours
Three partial circleson a white background
24
The binding problem. Solution 1

• Cardinal (grandmother) cells
• Object-selective cells that are sensitive to the
  configuration of features characteristic of the
  object
• Disadvantage requires a huge number of cells

25
Jennifer Aniston as a grandmother?
Quian Quiroga R., et al. Nature, 435. 1102 - 1107
(2005).  Invariant visual representation by
single neurons in the human brain
26
The binding problem Solution 2 (FIT)
Attention window
Aha! A red cube moving right!
Location map
shape map
Colour map
Motion map
retina
Homunculus
The output of the attention window is matched
with stored object representations
27
Neural realization of solution 2

• temporal binding through synchronous firing

shape map
Colour map
Motion map
retina
28
The Homunculus Coincidence-detector cells?
early-phase cell
late-phase cell
29
Back to grandmother

• Coincidence detectors are cardinal cells
  grandmother cells
• Therefore, convergence onto coincidence detectors
  is unlikely
• Maybe, synchronized neurons act as distributed
  representations that activate target (output)
  sites directly
• Goes in direction of a global workspace (Baars)

30
Consciousness requires integration
NCC What are the neural mechanisms, which map
the unified contents in phenomenal consciousness
to corres-ponding neural entities in the brain?
31
Structure for Consciousness
32
Binding and neural correlates of consciousness

• Neural mechanisms of binding
• What are the Neural Correlates of Consciousness
  (NCC) ?
• Special neurons ?
• Like grandmother cells
• Single neurons, firing rate ?
• Cell assemblies, synchrony ?
• Plausible

33
The binding problem

• Part 2

34
Temporal coding

• Temporal coding as a potential solution of the
  binding problem
• Assumption
• Features of one entity are bound by synchronous
  neuronal discharges features of different
  entities discharge in an uncorrlated manner
• Experimental evidence (selected references)
• Gray et al., (1989), Nature, 338, 334-337
• Engel et al., (1991), Science, 252, 1177-1179
• Kreiter Singer (1996), J Neurosc., 16, 2381-2396

35
The Binding Problem Neural coding I
36
The Binding Problem Neural coding IIProposed
solution Temporal coding
37
Hypotheses
38
The temporal binding model(von der Malsburg,
1981 Abeles, 1982)
39
Consciousness requires selection
Shepard, 1990
40
Temporal binding and selection(Engel, Fries
Singer, 2001)
41
Experimental approach (Singer, 1999)
42
Synchronous oscillations as the neural substrate
for feature bindingSelected experimental
evidence
43
Synchronous oscillations and feature
binding(anesthetized cats, Gray et al., 1989)
Stimulus related feature binding is reflected in
the temporal code, NOT in the firing rates!
44
Synchronous oscillations and arousal(Munk,
Roelfsema, Koenig, Engel Singer, 1996)
45
Human data Working memory IHuman EEG study
(Tallon-Baudry Bertrand, 1998)
46
Working memory IIHuman EEG study (Tallon-Baudry
Bertrand, 1998)
47
Temporal binding and the NCC - Postulates
I(Engel Singer, 2001)
48
Temporal binding and the NCC - Postulates
II(Engel Singer, 2001)
49
Temporal binding and the NCC - Postulates
III(Engel Singer, 2001)
50
Binding and temporal coding - Summary(Engel
Singer, 2001)
51
But

• Some questions with and alternatives to the
  previous ideas and results

52
Synchrony does it make sense?

• Who is reading the synchrony?

action
shape map
Colour map
Motion map
retina
53
Why is there synchrony in the brain?
54
Many potential sources of synchrony

• External sources of synchrony
• eye movements
• common input
• Internal sources of synchrony
• common input
• lateral connections, phase locking as an emergent
  property (fireflies)
• Strogatz, S. (2003). Sync. The Emerging Science
  of Spontaneous Order. New York, NY Hyperion
  Books.

55
External/Internal Common input
Each neuron is driven by 1,000 random inputs.
Individual pairs of neurons share 10 (a), 25
(b) or 50 (c) of those inputs.
Salinas, E. Sejnowski, T.J. (2001). Correlated
neuronal activity and the flow of neural
information. Nature Reviews Neuroscience 2,
539-550.
56
To bind or not to bind(1)

• Binding problem is a pseudo-problem (Valery
  Hardcastle)
• Choosing the appropriate level of analysis
• not individual neurons
• not small neural circuits
• but the dynamics of large neural networks
• chaotic attractors in the brain (Freeman)
• Microscopic to macroscopic
• (seemingly) random microscopic behavior (cell
  firing) may nevertheless underlie ordered
  macroscopic behavior
• Compare An ant colony
• individual ants seem to behave in an unstructured
  way
• the colony as a whole exhibits highly-structured
  behaviour

57
To bind or not to bind (2)

• Sensory-motor coordination as the basis of
  perceptual experience
• the binding problem is actually a
  pseudo-problem a by-product of the, in our
  opinion erroneous, conception that visual
  experience is generated by activation of brain
  mechanisms (ORegan and Noë, 2000).

58
Final remarks
59
Temporal order (simultaneity)

• Computational point of view
• Neural spikes are highly localized in time
• In general, simultaneity leads to fast responses
• Spatial and temporal simultaneity may have
  significance for computational efficiency only
• Spatial and temporal simultaneity inside the
  brain are only meaningful to an external observer
  (e.g., neurobiologist), not for the conscious
  agent itself!