REVERB Sheffield update

1
Integrative computation for autonomous agents a
novel approach based on the vertebrate brain
http//www.shef.ac.uk/abrg/reverb/public/
K.Gurney_at_sheffield.ac.uk
2
The problem of action selection
Fight, flight or feeding, but not do nothing
3
Action selection formalised
Animals can integrate a massive array of sensory
and state information to guide complex motor
actions to effect goal directed behaviour
Computationally, compare with real time OS
Is there a biological real time OS?
4
Specific problems one motor plant and visual
sensory input
Specific forcing domain is the gaze control
system driven by a variety of visual sensory
input
Potential saccades
5
Specific problems multiple motor plant and
visual sensory input
Learning Adaptation Plasticity
6
The remit of the project

• It is not a project on vision (object
  recognition, texture discrimination, stereo,
  structure-from-motion, etc.)
• It is not a project on motor control (complex
  manipulation etc)
• It is a project looking at the fundamental
  computations that enable sensory information to
  govern actions and behaviour.
• The first two are like applications running on
  the real time OS

7
Outline of proposed solutions

• Layered architectures

8
Generic plan of the vertebrate brain
Based on Butler and Hodos, 1996
9
Pure cognitive system (PCS)non-layered system
10
Problems with the PCS

• Real cognitive agents (e.g. humans), posses
  internal biological imperatives or drives (e.g.
  reproduction, feeding, survival from physical
  threat) which might necessitate regulation of the
  PCS...

11
PCS augmented with control
Most animals get by without a complex PCS -
perhaps it evolved as a secondary enhancement to
survival (via intelligent interaction with the
environment) Normans chapter in Issues in
Cognitive Modelling, by Aitkenhead.
12
Low level regulation as primary
Regulatory system
Sensory stimuli
Physical actions
Is it limited to two layers?.
13
Solutions layered architectures and basal
ganglia arbitration
Basal ganglia
Arbitration
motor actions
14
Layered architectures in brains and robots
Prescott et al. 1999
15
Outline of proposed solutions

• Computation at many levels of description

16
Computation in the brain
Gurney et al., (2004) Trends Neurosci 27,
453-459.
17
The basal ganglia input a computational nexus
18
Methodologies

• We will bring to bear a range of disciplines and
  methodologies

19
Computational neuroscience

• Build biologically realistic models of a brain
  system which are constrained by neuroscientific
  data at the relevant level of description.
• Compare biologically inspired v biologically
  constrained
• The key computations may be deeply embedded in
  the specifics of representations, neuronal signal
  processing etc

20
Robotics 1

• Embodied computational neuroscience provides a
  strong test of a model
• Closes the environment-agent-environment loop,
  giving realistic sensory input sequences.
• Forces consideration of architectural issues not
  apparent otherwise

21
Robotics 2

• Robotic implementation of the computations in
  more well engineered devices, promises robots for
  real-world applications

22
Real time processing

• Working with real-time dynamic environments in
  the embodied models will require real-time
  processing
• While powerful conventional computation
  facilities may help, we will do best to take
  advantage of custom hardware to
• optimise our chances of achieving real time
• allow low power embedded solutions

23
Mathematical and computational analysis

• An abstraction of the underlying computations we
  discover will form the novel computation
• This abstraction may take the form of classical
  algorithms OR neural network mechanisms.

24
If you have built castles in the air, your work
need not be lost. That is where they should be.
Now put the foundation under themHenry David
Thoreau

• Some work so far

25
A first layered architecture saccadic control
26
Replication of psychophysical data
Kalesnykas and Hallett, 1994
Findlay and Gilchrist, 2003
Data
Reaction time (ms)
Target eccentricity (deg)
Normal
Parkinsonsdisease
Model
Reaction time (ms)
Target eccentricity (deg)
27
Computations for decision making

• MSPRT
• Asymptotically optimal
• Among all tests allowing certain error
  probability, MSPRT requires minimal number of
  observations as the error probability ? 0

Dragalin, Tertakovsky, Veeravalli , 1999
28
Basal ganglia implement MSPRT
29
Predictions
exp
linear
Figure kindly provided by Atsushi Nambu
Hallworth, Wilson, Bevan (2003) Wilson, Weyrick,
Terman, Hallworth, Bevan (2004)
30
Learning action-outcome pairs

• Two requirements
• repeat action and elicit unexpected event/outcome
  (bias the training set)
• Use repeated occurrence to establish association
  (world model)

Hypothesis 1) is done via the BG and
colliculus (Nature Reviews Neuroscience)
31
Static feature salience maps
Eye movements are directed to locations of
highest saliency
These saliency maps are constructed from
luminance, colour, contrast, edges
32
Immediate future

• New ideas on learning
• Action outcomes, habituation
• Immediate Milestones
• Augment basic gaze control model to include
  static salience maps
• Implement in hardware 1 pan-tilt head with
  conventional camera
• Implement in hardware 2 static feature maps in
  existing AVLSI camera system (SCAMP chip)
• Establish basal ganglia control of robot arm

33
END

• For now

34
Computational infrastructure