Rachel Wurzman

1
How the Cortex Works

• Rachel Wurzman
• In 26.08 minutes

2
Just so you know

• Neurons dendrites, soma, axon, synapse

3
Invariant Representations

• Information flows up and down
• Collective activity on a bundle of fibers is a
  pattern
• By the time get to the top layer, cells fire
  whenever an object is present
• Network of feedback connections (more than
  forward!)
• Prediction requires a comparison between what is
  happening and what you expect to happen

INVARIANT REP
shape
color
Retinal spot
4
Invariant Representations

• Happens up and down each sense, up to association
  areas which are between senses

5
Invariant Representations

• Transformation from specific to invariant occurs
  in all sensory areas of cortex
• Pressure ? sounds ? words ? phrases
• This is a pen. A what? A pen. A what? A pen. Oh!
  A pen!
• The neural activity corresponding to the mental
  perception of objects lasts longer than the
  individual input patterns.
• Higher in cortex, fewer changes over time

6
Integrating the senses

• Something I hear can lead to a prediction of what
  I see. (cat, slamming the book)
• Surprised if it isnt what you expect

7
Integrating the senses

• ALL SENSORY AND ASSOCIATION AREAS ACT AS ONE a
  multibranched hierarchy
• All predictions are learned by experience (Its
  all just a little bit of history repeating)
• The motor cortex behaves like sensory system
  downward flow in motorcommands
• All one sense seeing, hearing, touching, and
  acting are profoundly intertwined

8
A new view of V1

• Why should IT be the only region with Invariant
  Representations?
• V1 should be seen as made up of smaller
  subregions
• The role of each higher region is to memorize
  patterns of the lower regions

• Think of left V1 and right V1 as separate sensory
  streams that get united higher up, just like
  senses on a broader scale
• Now each layer of cortex forms invariant
  representations of the input from lower areas

9
A model of the world

• Nested or hierarchical structure Notes ?
  intervals ? phrases ? melodies ? songs
• Sub-objects Line segments ? shapes ? noses ?
  faces ? person
• Only exist in one moment in time, but hierarchy
  allows you to extrapolate the permanent details
• Sequences! Follow each other in time, if not
  always order. (facial recognition ex.)

10
A model of the world

• Predictability is the very definition of
  reality. If a region of cortex finds it can
  reliably and predictably move among these input
  patterns using a series of physical motions and
  can predict them accurately as they unfold in
  time, the brain interprets these as having a
  causal relationship.
• Brain stores sequences of sequences

11
Sequences of sequences

• Unfolding of sequences (Preamble)
• The same amount of detail in the feedback applies
  to each level
• The exception to this is if the lower regions of
  cortex fail to predict what patterns they are
  seeing, they consider this an error and pass the
  error up the hierarchy- until something
  recognizes the pattern

12
Sequences of sequences

• The brain must classify patterns (paper example)
• The cortex is flexible in its pattern
  classifications
• Difference between your brain and a machine it
  recognizes sequences of patterns that correspond
  to the world, as opposed to matching objects with
  prototypes. The sequences are reality.

13
What a region of cortex looks like

• Primary sensory areas are the largest, but
  remember they have subdivisions

14
What a region of cortex looks like

• Vertically aligned cells in a column react to the
  same stimulus one column may respond to lines
  like this / others \.
• Activity in layer 4 causes layers 2 and 3 to
  become active, which then trigger 4 and 5.
• Columns can also refer to groups that form from
  one progenitor
• 90 of synapses on cells come from different
  columns
• Moncastle believed that the cortical column is
  the basic unit of computation in the cortex
• For a column to predict when it should be active,
  it needs to know whats going on elsewhere

15
What a region of cortex looks like

• UPWARD FLOW Converging inputs arrive at layer 4.
  The form a passing connection in layer six on the
  way. 4 sends axons to 2 and 3, which each send
  axons up to the next higher region.

16
What a region of cortex looks like

• Downward flow Layer 6 cells from above flow down
  to layer 1 of hierarchically lower regions. Layer
  one spreads this input out, picked up on by
  dendrites of layers 2, 3, and 5.
• Axons from 2 and 3 form synapses in 5 and excite
  cells in 5 and 6

17
What a region of cortex looks like

• Layer 1 contains information about which columns
  were just active in the cortex
• Layer 5 cells output to motor processes in M1,
  and also in sensory areas
• Axons from 5 split in two one outward and one to
  the non-specific thalamus
• From thalamus, they loop back to areas layer 1

18
What a region of cortex looks like

• Like the delayed feedback that lets
  auto-associative memories form
• Higher regions of cx spread activity across layer
  1 in lower regions
• Columns within a region spead info across layer 1
  of same region via Thalamus
• Layer 1 Sequence name, and where we are in the
  sequence

19
The details

• Converging patterns going up, diverging going
  down, and delayed-feedback through the thalamus
• INHIBITION when a layer 4 cell of a column
  fires, it classifies the input as its own.
  Inhibitory cells prevent others around it from
  firing too.

20
The details

• 1) Layer 4 cells fire
• 2) This causes 2, 3, and 5 to fire too
• 3) If synapses of 2, 3, and 5 in layer one are
  active at this time, synapses get strengthened.
  FIRE WIRE SYNCH LINK.
• 4)Now, synapses of 2, 3, and 5 in layer 1 can get
  those cells to fire even without Layer 4 input
• Layer 1 can now detect patterns via memory

21
The details

• The cells now fire in anticipation when they see
  a pattern at the synapses
• They get the name of the sequence in layer 1
  from higher regions now info in layer 1
  represents the name and last item in the sequence
• Cells want information that will predict when
  they will fire from info from below

22
The details

• Cortex needs a way to keep the input to the next
  higher regions constant
• Answer inhibition of layer 2 and 3 cells when a
  column predicts its activity, activates them when
  it cant

23
The details

• Layer 2 cells become active when within the
  sequence name.
• 3b only active prior to learning
• 3a has dendrites in layer 1, and when sees a
  pattern, inhibits 3b
• Layer 2 driven purely from higher cortical
  regions they synnapse with layer 6 from above,
  and project back to the higher region to form
  stable activity
• Sum cortex learns sequences, makes predicitions,
  and forms invariant representations for the
  sequences.

24
The details

• We combine feed-forward information (actual
  input) with feedback information (a prediction in
  invariant form) to make predictions about new
  events
• Ex region of cx expects a fifth. All layer 2
  cells representing fifths active. Layer 4 cells
  with the note just heard is active. The
  intersection between these two columns represents
  the column of interest.
• Think holes in a paper lining up active 2 or
  3invariant prediction, with columns with active
  4 input from below.

25
The details
26
The details

• Finally, when layer 6 cells project to layer 4 in
  their own column, the predictions become the
  input
• Folded feedback or imagining
• What we see is dependent on our actions we must
  know what actions we are undertaking to predict
  what comes next
• Motor behavior and sensory perception highly
  interdependent perception and behavior almost
  one and the same

27
The details

• Layer 5 cells that project to the thalamus and
  back to layer 1 also project to motor layers of
  the old brain.
• Thus, what just happened both sensory and
  motor is available in layer 1.
• Motor behavior must represent a hierarchy of
  IRs.
• Generate movement by thinking of its IR
• Ex moving to kitchen, saccades on faces.

28
Flowing up and flowing down

• When an unexpected pattern arrives
• 3b keeps passing information up to next higher
  region, involving more and more of the cortex,
  until it is recognized. Then, it is passed back
  down.
• Like going into a foreign country, or finding a
  pattern in a picture. Errors get passed up until
  new patterns are learned and sense is made. Then,
  back down.

29
Can feedback really do that?

• Prevailing opinion is that far away synapses are
  only modulatory
• Hawkins disagrees for memory-prediction model to
  work, far away synapses must act as coincidence
  detectors and cause a cell to spike

30
How the cortex learns

• Input changes from recognizing individual
  patterns to groups of patterns
• When bottom-up inputs become more object
  oriented, it frees higher regions for more
  complex pattern making
• The MEMORY of sequences moves lower and lower
• Ex reading
• This is why young brains are slower and geniuses
  faster
• Hawkins would like you to know that he is such a
  genius, but, then why cant he integrate the BG
  into his model?

31
The hippocampus on top of it all

• Basal ganglia as primative motor system,
  cerebellum as precise timing of relationships and
  events, hippocampus stored memory of specific
  events and places
• Hippocampus learns long-term memories and sends
  the info to the cx
• Hippocampus as at the top of the cortical
  structure

32
The hippocampus on top of it all

• Unexpected patterns get passed higher and higher
  up so truly new experiences will reach the
  hippocampus
• Aging example fitting into past memories so less
  and less is seen as new
• Unlike the neocortex, the hippocampus has
  heterogeneous structure with specialized regions.
  
• Only form permanent memories if experience it
  over and over, in reality or by thinking of it,
  so that it gets passed down to cortex.

33
An alternate path up the hierarchy

• Info that gets passed up also goes up via an
  indirect pathway through the thalamus thalamus
  only passes info up.
• Speculation the thalamus serves to direct our
  attention to details lower in the hierarchy
• Imagination
• It bypasses the grouping of sequences in layer 2,
  sending raw data to the next level of cortex
• If input to alternative pathway is strong enough,
  sends a wake up signal to the higher region,
  which can then turn on the pathway
• Attention directs this? Unexpected stimuli.

34
Closing thoughts

• Many of these thoughts, especially that revolve
  around the memory-prediction model, are
  speculative and we neuroscientists know its a
  gross simplification
• Ideas may change
• Big task, but getting a program on a computer to
  work instantaneously was huge too
• Our intuitive sense of the capacity of the
  cortex, with its billions of neurons and
  trillions of synapses, and the power of its
  hierarchical structure is inadequate.
• But we can build machines that do it.