How do Brains Talk

1
How do Brains Talk?

• Patrick Hudson

2
Structure of the Talk

• The problem
• The specificity of human language processes
• Models of language processing
• Linguistics
• Cognitive Psychology
• Computational models
• Testing models
• Descriptive adequacy
• Explanatory adequacy
• Conclusion

3
Introduction

• The question - How is Natural Language processed
  by brains?
• More specifically What brain mechanisms allow us
  to speak and hear, to talk and understand?
• How should we study this?

4
Is Language Special?

• Language is just a special way of supporting the
  communicative process
• Everything communicates
• Humans
• Horses
• Honey bees
• Only humans can make new references out of old
  material - endlessly
• Syntax supports this generative ability
  infinitely
• Great apes, despite genetic similarities, cant

5
Modelling Language

• Linguistic models - what is it?
• Descriptive
• Formal
• Cognitive models - what does it do?
• Box models
• Connectionist models
• Computational models - how does it do it?
• Automata models
• Neural models

6
Linguistics

• Descriptions of the components (N, V, mod etc)
• Grammars (syntactic, morpho-phonological etc
• Semantics
• Bases are rules of combination
• Formal systems define what is computed
• Competence
• Limits on learnability
• But not how
• Performance

7
Cognitive models

• Boxes with components and connections
• Labels often mapped directly onto linguistic
  categories
• Assumptions about how the boxes work are usually
  vague/implicit
• Predictions are based on generating RT differences

8
Problems with cognitive models

• Wont work - they are meant to model RT data
• Cant be attached to other boxes even if they do
  work - only interested in own box
• Couldnt learn language even if they did work
  together - only intended to do one thing
• Can be implemented in brain hardware - too hard
• Dont make sense even if framed as neural models
• Have become too hard to test within conventional
  paradigms

9
Computational models

• Intended to actually perform
• Classically based on linguistic models or their
  formal equivalents (ATNs)
• Have to confront issues of what, where, how and
  when
• There are no magic boxes allowed
• Connectionist models still often are just
  neuralised conventional models

10
Functional specifications

• Real time operation
• Process from left to right
• Cover both speaking and hearing
• Non-English specific
• learnable
• Share common components
• Operate with sensible components
• Build a model meeting these and you have no
  explanatory adequacy, only descriptive adequacy
  and internal consistency arguments
• In short, hard to test with more of the same

11
Failure modes

• Produce slips and hesitations
• Match aphasic phenomena
• Demonstrate standard pathologies (dyslexia etc)

12
David Marrs levels

• Computational level - what is it we compute?
• Algorithmic level - what can we compute?
• Implementational level - how does it actually get
  done?
• Marr argued that you need to go top down
• The problem is the implementation level may well
  determine what is possible to compute

13
How to test models

• Linguistic tests are generally still based upon
  some notion of acceptability and match against
  intuitive judgements
• Cognitive models are based on RT, but this serves
  as a discriminator, the times as such mean
  nothing
• Connectionist models can make sense of times and
  failures/intrusions but are criticised for having
  to many degrees of freedom
• My original models used failure modes as the test
  set

14
Engineering

• Science is about properties
• Engineering is about functions
• The relationship is one of mutuality and the
  boundaries are sometimes unclear
• This applies in psychology and computer science
  as well as physics and chemistry
• Understanding brains requires a lot of
  engineering
• We need articulated models of how the machinery
  works

15
Justification in model construction

• Take a set of criteria and construct a model that
  will, a priori, meet those criteria
• The problem is that none of this therefore
  enables you to explain why you have what exists
  in the model
• The model therefore needs to be tested to see if
  it can generate data that was not used to justify
  the original design

16
Brain scanning

• Modern psychology uses MRI, fMRI, PET scans etc
• Thses are supposed to form the breakthrough in
  science
• The information is still upgraded localisation
  data, identical in effect to that obtained from
  studies of neurological patients
• Nothing new has been discovered since Kleist
  (1917) and Head (1924)
• A major problem is that they do not use any
  models of what the machinery actually does
• This level of detail is provided by computational
  models

17
What to look for in a brain?

• Time resolution often too slow
• E.g EEGs
• Space resolution too coarse
• No link to the cytoarchitectural differences
• Interpretation of events driven by a model of
  brain function
• Models similar to Lichtheim (1887)
• Most models of pathology are re-labeled symptom
  boxes
• What is required is an interpreted functional
  model of brain function, not a restatement of the
  problem

18
Conclusion

• Linguistic models operate at the computational
  level
• Congitive models operate, at best, at the
  algorithmic level
• Computational models allow us to understand what
  really goes on and, integrating the other two,
  provide the best explanatory adequacy