Chapter 12: Outputs of Visual Processing

1
Chapter 12Outputs of Visual Processing

• IT Inferior Temporal Visual Cortex
• LPF/PF (Lateral) Prefrontal Cortex
• RC Recurrent Collateral Connections
• Attractor Networks Feedback like system

2
We will discuss

• Ventral visual processing stream
• IT output provides distributed representation
  of. what
• (Parietal sends where info)
• Both IT P output to STM in what region
• PFC (Prefrontal Cortex)
• why STM system must be separate from IT P
  system
• interaction based on attractor networks

3

• There are both superficial and deep cortical
  neuron layers in each attractor network
• Superficial layers (s) project onto self or
  higher cortical level (like s for superman up up
  and away!)
• Deep layers (d) project onto self or downwards (d
  for downward)

4
How does STM work???

• Network must maintain firing rate after stimulus
• For how long?
• In a monkey IT, usually between 0.1 10 seconds
  in PF even longer
• How is it maintained? (also see figure)
• The collateral connections of (nearby) pyramid
  cells create feedback loops. These undergo
  associative modification.
• These collateral feedback loops are called
  recurrent collateral connections

5
Intramodular connections
PF
IT

• This model also says connections between networks
  must be weak.
• This parameter, g, indicates the relative
  strength of inter-modular to intramodular
  connections.

6
Intramodular connections
PF
IT

• Weak g is needed for PF to maintain firing rate
  (ie. a percept) while several intervening stimuli
  pass through the posterior IT networks

7
What is the Two Network Model ofthe visual-STM
system???

• Separate networks are needed for both perception
  and STM to work simultaneously. So where are
  they??
• Network in IT perceptual functions
• Network in PF for maintaining STM during
  intervening stimuli
• These networks are coupled ie. they communicate
  with each other with forward and back projections
• How do we know..

8
Stimulus Match to Sample Task

• Used to understand STM.
• Response in RC of PF is activated by both visual
  input of matching stimulus backprojected memory
  of sample stimulus
• Two networks firing in synchrony augment response
  choice

9
Stimulus Match to Sample Task

• This model says that for continuous perception to
  occur, there must be a module that maintains
  representation of the stimulus during intervening
  stimuli (ie. a STM must exist)
• Two network model allows planning which is
  dependent on ability to maintain several short
  term memories simultaneously.

10
Visual Search Task
11
Yummy, equations -p
This models how current (ie. firing rate) changes
in time

• Iai is current for neuron i in module a
• a IT, PF (ie. a is either IT or PF)
• Iai decreases in time with constant t,
• increases proportionally to the activity
• elsewhere in the network
• is the synaptic efficacy between two
  neurons. (ie. between neuron ai and presynaptic
  neuron j of module b.)
• vbj is the weight of neuron j of module b
  (ie. firing rate)
• is current from the stimulus that is
  external to network

12
This model computes synaptic efficacy (firing
rate correspondence) between neurons within a
module (either IT or PF)

• P is number of binary patterns of active neurons
  that are possible both for IT and PF
• h is a neuron with value 1 or 0 with
  probabilities f and (1-f)
• Nt is the normalization constant

13
Synaptic Modification

• This is process of setting up neurons to create
  an attractor which can hold an item as STM
• Once set up it may be reused when triggered by
  appropriate cue, even without further synaptic
  modification

14
Part II Visual Outputs to LTM

• IT projects to the Hippocampus to create LTMs
• In the hippocampus objects may be located as part
  of a spatial scenes. This is called episodic
  memory
• especially primates, not so much rats
• Primate object-memory representation could be
  generated by a continuous attractor

15
Damaged Hippocampus

• Anterograde amnesia. What causes this in humans??
• bilateral damage to HC and nearby parts of IT
  (Squire)
• Which LTM memories located in HC
• HC shown necessary to learn declarative memories,
  epsiodic
• Not spatial processing eg. An object-place
  memory task which requires memory of object and
  environment or context in form of snapshot
• LTM functions not affected by HC damage?
• procedural memory, anterograde amnesia does not
  impair procedural memory

16
Rats and Monkeys

• Rat HC pyramidal cells recognize a previously
  learned scene only from the same location and
  orientation egocentric (body based)
• Learning a new scene for rat HC pyramidal cells
  takes about 10 minutes
• A group of cells can map 2 spaces
  simultaneously, and is able to process/represent
  only one them
• Rats also have task related HC cells
• cells respond to olfactory stimuli with
  particular behavior response

17
Rats and Monkeys

• Primate HC has spatial view cells that respond to
  where monkey is looking, from any vantage point
  allocentric (world based)
• Idiothetic cues to trigger memory when scenes
  details are obscured
• eye position
• head direction
• linear and axial whole body motion

18
Why Study Spatial View Cells

• Primates have spatial view cells
• They enable object-place memory (a kind of
  episodic memory)
• some HC neurons respond to combo of object info
  and spatial info
• HC cells respond both to presented image and
  location for response this is called memory
  convergence (2 stimuli ? 1 memory)

19
Show figure

• Rats and Monkeys compared
• Rats Monkeys
• place cells spatial view cells
• wide visual foveate vision
• field
• Perhaps cells use same computational process and
  respond according to their different input
  apparatuses

20
Hippocampus Models

• CA3 Recurrent Collateral Connections create
  auto-association networks enabling. Every CA3
  neuron to associate with any other CA3 neuron
  involved in the same memory
• The number of different memories p in the CA3
  system
• RC recurrent connections
• a sparseness
• k synaptic efficacy
• Eg. For a rat CRC 12000 this translates to
  between 12k 36k memories
• Low sparseness (ie .02) ? more memories
• Same idea going between networks
• back projections down to CA1 ?

21
CA3 RC Network

• Mossy fibres help learning only, not recall

22
Continuous Attractors

• To represent spatial patterns ? use continuous
  attractor networks
• HC ? represents spatial patterns ? uses
  continuous attractor networks
• Continuous networks can explain how in the dark,
  rats can still maintain place cell firing (or
  even update the response with idiothetic inputs)
• Now known, attractors can store both continuous
  and discrete patterns
• locations in space continuous
• objects discrete
• HC receives and combines both input types
• For an event with both discrete and continuous
  aspects spatial info can be retrieved from
  object cues and vice versa

23
Cool?

• Thanks!

24
I am intrigued by Rolls et al.'s research about
specialized hippocampal cells (p.416). For
example, they find cells that respond to a
combination of which picture is shown and where
the response must be made. Cells like these,
that integrate perception of stimuli with what
response is needed, seem like they could be close
to feeding straight into consciousness (i.e.
sound homunculus-like). However, since these
cells are not tuned to respond to any particular
stimuli or to coordinate any particular
response, I can't get a handle on quite how they
could work. Is that known? -S.R.
25
Rolls and others have found neurons responding to
many strange stimuli that I should never have
thought of for instance, hippocampal cells that
register not only where the animal is (rat) but
where it is looking (primate) cells that respond
to the physical properties of a stimulus (sweet
vs. salty), others that respond to its reward
contingencies. We take as an article of faith
that every mental event has an accompanying brain
event.  Does every mental event have a
corresponding single-neuron event?  Perhaps yes,
if Barlow is right and grandmother cells exist. 
No, if many 'events' have a distributed code that
spreads them across many neurons. In the present
state of the art we can record events in single
neurons (with electrodes) and in rather coarsely
defined brain areas (with fMRI).  Between the
microscopic and macroscopic techniques lies a
mesoscopic gap - we cannot record events of the
size of cell assemblies.  Is this where most of
the brain action is? -S.A.
26
Meanwhile, what would we find in a catalogue of
every stimulus that is known to excite single
neurons in the brain? It would include faces,
localities, reward contingenciesŠ  Would it
contain every imaginable stimulus, and if not,
what would it leave out?  Of these left-out
items, would cell assemblies respond to them? 
I'm really wondering what we could conclude from
such a catalogue.  If it contains everything,
does it really tell us anything?  And do we know
of any instructive omissions from such a
catalogue? (dogs that did not bark during the
night). I've heard many informal criticisms of
Rolls' book  --   too concentrated on his own
work, ignoring the work of others in particular,
hard to read and hard to follow.  But what do we
think, now that we have (in theory) read the
book?  Is most of it "going nowhere", or is Rolls
really "on to something" that we ignore at our
peril?  Maybe his insights will give us huge
breakthroughs, that we lack the intelligence or
perseverance to descry?  What does everybody here
think? -S.A.
27
In section 12.4, the authors point out how
translation invariance in IT is a problem for the
issue of how information about an object's
spatial coordinates is made available to the
motor system. They propose that to solve this
problem, the motor system simply directs action
toward the object foveated, since in most cases
that is the object of interest. Although this
explanation does seem possible, I am still not
sold. I think a much more parsimonious
explanation would be that even after translation
invariance takes place in IT, some sort of
representation of the target's spatial location
is preserved within the system. Why should the
motor system start from scratch when information
about the spatial coordinates has already been
extracted, albeit at an earlier level of visual
processing? L.L.
28
1. The authors assume that thresholds are
exceeded. Is there any mention in the book about
how this may occur. 2. The authors assume there
is an "activation function." How might this be
derived from fundamerntal principles of neuronal
activity rather than just be a convenient
assumption that makes the model fit some
data? 3. The measure of "sparseness" is claimed
to have a value that can only occur in a very
extreme and limited citcumstance. Is "sparseness"
necessary? If so, what would be a better way to
compute it. 4. Multiplication occurs in
equations throughout the book. How can
multiplication occur, physically? 5. Signals
passing through the nervous system are
representable as independent stochastic random
variables. But when a threshold is posited the
sum of such signals is no longer a sum of
independent RVs because the sum is limited (ie
must at least equal the threshold). The
conventional algrbra of independent random
variables no longer applies. What effect can this
have on the theory? On any theory?
29
6. Although the signal transmission system of
the nervous system is known to be a stochastic
process, and therefore the algebra of probability
theory applies to models that represent the
nervous system, the book uses real numbers and
real algrebra and essentially linear system for
its modelling. Is there something very wrong
here? S.L.
30
Page 416  "These primate 'spatial view'
hippocampal cells encode information in
allocentric (world-centred, as opposed to
egocentric) coordinates." There are two ways in
which (for instance) a ship codes its own
position  visually, by measuring nearby visual
landmarks or from dead reckoning, by recording
and remembering all its own motions since it left
its home port.  Dead reckoning is subject to
cumulative drift, whilst visual sighting is
probably not.  (I ignore Harrison's chronometric
measurements of longitude, and GPS methods). 
Which of these two methods, or what combination
thereof, do (or could) these hippocampal cells
use? S.A.