Sha'a

1
On the sixth day of Creation...
2
after some clarification...
granulate and multiply And replenish the
GENESIS I, 28
3
granulate AND MULTIPLY ?!?
We took it to mean, to insert granule cell
layers into our cortex and then, to see if
it leads to multiplication...
4
GRANULATE, I .

• take the medial wall of the
• premammalian cortex

and insert the fascia dentata, with its granule
cells, at the input end
note that the granule cells have become
(excitatory) interneurons
5
(No Transcript)
6
(No Transcript)
7
Multiply, I ?
NO!
H
human
opossum

• the new structure remains stable and unique
  across mammalian species

H
8
GRANULATE, II

• We took it seriously, and went on,
• trying to insert granule cell layers
• into our cortex. Not quite in the
• same way as for the medial wall...

• now, the dorsal cortex. It acquires fine
  topography ...

• ...and it laminates

9
? granulating the dorsal wall, leads to the
mammalian isocortex

• the brand new neocortex has laminated, i.e.
  inserted a granular layer IV in between two
  pyramidal cells layers.

what does this other granulation buy us?
Layer IV granules are now (excitatory)
interneurons
10
Isocortical lamination

• emerges together with fine topographic mapping
• does not apply to the non topographic olfactory
  system
• is underdeveloped in caetaceans
• It might be a computational solution to the
• need to relay precise information about
• both where and what sensory stimuli are.

11



the model
src
recurrent collaterals






patch of cortex input station




feedforward connections
sff
input activity spatial focus detailed pattern
R
12

• The activation of units in the previous station
  is the product of a spatial focus, say, a
  Gaussian of radius R (which presumably would be
  picked up by optical imaging, or by multi-unit
  recording) and a detailed unit-by-unit pattern of
  activity (which would require single unit
  recording to be revealed). p patterns of activity
  (e.g. 2-12) are established at the beginning,
  drawn at random from a given distribution, and
  used repeatedly in one simulation.
• The activation of units in the cortical patch is
  compared with the activations resulting from the
  application of each input pattern at each spatial
  focus, to decode the pattern ? and focus x of the
  current activation. This allows measuring
• as well as
• both population measures, reflecting activity in
  the whole patch

13

• Both recurrent and feedforward weights are
  modified according to a simple Hebbian
  associative rule, over the course of several
  training epochs. Each training epoch involves
  presenting, in random order, each input pattern
  at each activation focus. The map is thus
  pre-wired at a coarse, statistical level, and
  self-organized at a finer scale.
• After a training epoch, noisy versions, again of
  each pattern at each activation focus, are
  presented for testing, with no weight change. The
  full information about position and identity
  cannot be decoded from the activation in the
  patch, because the activation in the input is
  noisy (in practice, e.g. 40 of the input units
  follow the prescribed pattern, and 60 are
  randomly activated with the same distribution)
• If R ltlt Src, it is rather intuitive to predict
  how much information can be relayed by
  feedforward projections of spread Sff

14

• Iident is small initially
• grows with learning
• no difference between layers
• Results for p4

15

• Ipos is less affected
• by learning
• decreases with more
• diffuse feedforward
• connections
• again, no difference between layers

16

• These data, plotted
• as Ipos vs. Iident,
• demonstrate the
• what/where conflict
• as a boundary
• using more patterns merely shifts the same
  boundary upwards

17
Differentiating a granular layer (IV)

• in which units receive focused FF connections,
  also more restricted RC connections, and follow a
  specific dynamics
• may nail down the focus of activation within the
  cortical map (preserving detailed positional
  information)
• without interfering with the retrieval of the
  identity of the specific activation pattern
  (achieved mainly by the collaterals of the
  pyramidal layers)

18



the model
src
recurrent collaterals






patch of cortex input station




feedforward connections
sff
input activity spatial focus detailed pattern
R
19

• Indeed it happens!
• Laminated cortex can
• relay more combined
• what and where
• information than if it
• were not laminated
• The advantage is somewhat more evident for larger
  p
• it is small, but should scale up in a network of
  realistic size

20
Dependence on the size of the cue the effect of
learning...
21
the advantage is there whatever the size of the
cue
22
but what do I do to layer IV ?
1) restrict its collaterals
2) focus its afferents
3) sustain its dynamics
(but suppress it in training)
23

• The network dynamics reflects the formation of
  attractors.
• Considering the distribution of activity of one
  unit across network states, before any learning,
  and at the beginning of an iteration cycle, it
  resembles a standard distribution seen for each
  network state, across different units.
• After learning at the end of the cycle, instead,
  many units fall into either a quiescent state, or
  a state with rather fixed activity, only mildly
  modulated by the net.

24
The granular layer

• may nail down the focus of activation within the
  cortical map (preserving detailed positional
  information)
• without interfering with attractor-mediated
  retrieval of the identity of the specific
  activation pattern (achieved mainly by the
  collaterals of the pyramidal layers)

25

• A differentiation between supra- and
  infra-granular layers may be usefully coupled to
  their different extrinsic connectivity, if
• the supragranular layers preserve both positional
  and identity information, and trasmit it onward
  for further analysis
• the infragranular layers relay backwards and
  downwards identity information freshly squeezed
  from the attractors, without bothering to
  replicate positional information

26
and what do I do to layer V ?
4) remove its afferents from layer IV
V
III
IV
27

• Laminationdirectional
• connectivity make
• each layer convey a
• better mix of
• information, beyond
• the capability of any
• unlaminated patch,
• whatever its Sff
• they also slow down learning, though, so the
  advantage would be greater if more learning
  epochs had been allowed (here they are set to 3)

28
(No Transcript)
29
Oops! I forgot the timing..

• ..this account is roughly independent of dynamics
  (a detailed analysis of relative timings, e.g. of
  the different inputs to the deep layers)
• the only dynamical element introduced is firing
  frequency adaptation, which is however used in a
  time-independent fashion
• we shall discuss more time-related uses of
  adaptation over the next two days, in generating
  transitions along continuous and among discrete
  attractors.

30
A functional hypothesis

• A common mode of operation of the primordial
  sensory neocortex of mammals may have been
  autoassociative attractor dynamics.
• Attractors may be formed by self-organizing
  weight changes on FF and RC connections, and may
  dominate the dynamics of both SG and IG layers,
  although the former can be kept in tighter
  positional register by layer IV.
• Thanks to Hamish Meffin, with whom I discussed
  such ideas, with divergent conclusions (see his
  Ph.D. Thesis, U. of Sidney)

31
2 suggestions

• Understanding specific mammalian mechanisms of
  information representation and retrieval may
  require quantitative (information theoretical)
  analyses at the level of populations of
  individual neurones
• Only notions of sufficient abstraction and
  generality as to apply to each sensory cortex can
  help explain the appearance, in evolution, of
  this universal neocortical microchip.

32
Multiply, II ?
YES !
cat
but why ?
hedgehog
monkey
We are busy trying to understand it.
Maybe next time...