CS 182 Lecture 28: Neuroeconomics

1
CS 182Lecture 28 Neuroeconomics

• J.G. Makin
• April 27, 2006

2
Decisions, Uncertainty, and the BrainPaul
Glimcher (2003) MIT Press

• Thesis neuroscience has been dominated by the
  reflex paradigm
• Alternative investigations rooted in economics,
  evolution, game theory, and probability

3
(No Transcript)
4
Reflex Theory

• Model Input-Association-Output
• (think of trying to explain language this way)
• Even ANNs?
• Methodology thoroughly constrain the environment
• Isnt this how science is done?
• Obscures a system-level view
• Has this really led researchers astray?
• Why are there so many questions on this slide?

5
Glimcher 2003
6
Reflex Theory (cont)

• Challenges to naïve reflex theory
• T. Graham Brown and the Half-Center Oscillators
  This is not the name of a band, as far as I
  know, though it should be
• Sherrington stimulus for walking from
  enteroceptive or interoceptive sources only
• Reafference and Efference Copy (Von Holst and
  Mittelstaedt)
• Glimcher actually has these confused

7
(No Transcript)
8
Reflex Theory (cont)

• Challenges to naïve reflex theory
• T. Graham Brown and the Half-Center Oscillators
  This is not the name of a band, as far as I
  know, though it should be
• Sherrington stimulus for walking from
  enteroceptive or interoceptive sources only
• Reafference and Efference Copy (Von Holst and
  Mittelstaedt)
• Glimcher actually has these confused

9
(No Transcript)
10
An Alternative

• Behavior is structured
• by goals (cf. shoulder reflex)
• by optimization strategies in the face of
  uncertainty
• Specification of the problem on the basis of
  function rather than implementation (Marr)
• In particular, the problem is an optimization
  problem
• Conclusion Neuroscience needs probability
  theory, economics, evolutionary theory, and game
  theory

11
Reflex Theory (cont)

• What reflex theory doesnt address
• the shoulder reflex (Paul Weiss)
• foraging
• mate selection
• exploratory behaviors
• Language thought

12
(No Transcript)
13
An Alternative

• Behavior is structured
• by goals (cf. shoulder reflex)
• by optimization strategies in the face of
  uncertainty
• Specification of the problem on the basis of
  function rather than implementation (Marr)
• In particular, the problem is an optimization
  problem
• Conclusion Neuroscience needs probability
  theory, economics, evolutionary theory, and game
  theory

14
(No Transcript)
15
I Optimization

• Q Optimization with respect to what?
• A Inclusive fitness but modularized. Evolution
  provides the goals, economics the optimization
  techniques
• Do we have a prayer at specifying the optimum?
• Phototransduction near the quantum limit
• Hair cells can detect individual fluid molecule
  collisions
• Convergent Evolution Cichlid fish of Tanzania

16
(No Transcript)
17
II Uncertainty Epistemological

• Reflex theory dominated by deterministic
  responses to input (from a highly constrained
  set)
• Alternative in general, we suffer from
  epistemological uncertainty, so we have to
  optimized in an indeterminate world

18
Uncertainty (cont)

• An empirical test of foraging economics
• the prey model, Parus major
• View foraging as an optimization problem choose
  the probability p_i of attacking the prey i that
  maximizes the rate at which energy is gained
• Solution
• zero-one rule
• independence from encounter inclusion rate
  principle

19
(No Transcript)
20
Uncertainty (cont)

• Frequencies of large and small mealworms were
  varied
• Small mealworms always had larger handling time
• Prediction (from optimal soln)
• Preference for large worms as their freq.
  increases, regardless of small worm freq. (by
  IEIR principle)
• If the bird couldnt get all the worms, it should
  give up entirely on the small ones (by the
  zero-one rule)
• Result yes and no (only 85 selective)
• Maybe this is an optimal strategy after all

21
Epistemological Uncertainty the BrainA Series
of Studies

• Input-association-output model
  sensory-parietal-motor
• Lateral intraparietal area (LIP) and monkey
  saccades
• Monkeys trained to perform task w/juice reward
• Invariant to input stimulus (light or button or
  whatever)
• Position-encoding
• Conclusion command signal (Mountcastle)

22
Epistemological Uncertainty the Brain (cont)

• Lateral intraparietal area (LIP) and monkey
  saccades
• Fixation and saccade tasks w/eccentric light
• Weak activation on fixation, but increasingly
  active over trials of saccade task
• Conclusion attentional enhancement (Goldberg)

23
(No Transcript)
24
Epistemological Uncertainty the Brain (cont)

• Lateral intraparietal area (LIP) and monkey
  saccades
• Memory saccade task target is extinguished but
  LIP neuron still firesuntil the motor command is
  executed
• Conclusion motor intention (Gnadt Anderson)

25
(No Transcript)
26
Epistemological Uncertainty the Brain (cont)

• Platt Glimcher encoding the probability of
  pay-off

27
Epistemological Uncertainty the Brain (cont)
Probability experiment
28
Epistemological Uncertainty the Brain (cont)
Value experiment
29
(No Transcript)
30
III Irreducible Uncertainty Game Theory

• Static environment ? Dynamic competition with
  other agents
• Then the optimal approach is given by game-
  theoretic approaches
• In these cases, the optimum often involves
  (purposefully) random behavior

31
Uncertainty Game Theory (cont)

• Example 1 Chicken

32
(No Transcript)
33
Uncertainty Game Theory (cont)

• Conclusion Smith is best served by behaving
  non-deterministically, but with probability 0.647
  of being a chicken. (Ditto for Jones.)
• If Jones finds non-randomness in the distribution
  of Smiths choices, he can predict above chance
  which option Smith will pickand win.
• Random behavior is the optimal solution, so we
  shouldnt expect behavior to look deterministic
  (contrast w/reflex theory).

34
Intermezzo How Random Are We?

• Paper, scissors, rocks
• Dice, viscera divination, etc. technological
  breakthrough (Jaynes)
• Unconscious vs. conscious behaviors natural
  selection vs. rational actors
• Pigeons, babies, and adults the matching rule
  and cognitive load (and reward)

35
Game Theory and Ethology

• Duck foraging
• Two feeders at opposite ends
• 33 ducks
• Rate of food depends on feeder, but the more
  ducks in an area the worse it is
• Where to sit?

36
Game Theory Ethology (cont)
37
Game Theory Ethology (cont)

• Person 1 2-gram bread ball every 5 sec
• Person 2 2-gram bread ball every 10 sec

38
Game Theory Human BehaviorWork or Shirk
39
Game Theory Human BehaviorWork or Shirk
(cont)
Insp -50
Insp -5
40
Game Theory Human BehaviorWork or Shirk
(cont)

• Experiment subjects play against a computer
  program which looks for statistical regularities
  in its opponents plays and tries to exploit them
• Subjects are only told that they can make money
  by playing
• 150 trials, then the pay-off matrix switches
  (unannounced)
• Guess how human beings played.

41
Game Theory Human BehaviorWork or Shirk
(cont)

• 150 trials, one pay-off matrix, vis-à-vis the
  Nash equilibrium?

42
Game Theory Human BehaviorWork or Shirk
(cont)
43
Game Theory Human BehaviorWork or Shirk
(cont)

• Work-shirk-work-shirk yields 50 behavior.
  Shannon entropy of choices?

44
Game Theory Human BehaviorWork or Shirk
(cont)
45
Game Theory Human BehaviorWork or Shirk
(cont)

• Switching between pay-off matrices?

46
Game Theory Human BehaviorWork or Shirk
(cont)
47
Game Theory the Brain

• Repeat the game, this time with monkeys instead
  of humans
• Simultaneously record from parietal area LIP
• Prediction if these neurons encode expected
  utility, then they will fire at constant rates
  over various movements and various rewards
  (contrast Platt Glimcher 1999)
• Now we have an experiment that yields
  non-deterministic behavior but about which
  predictions of lawful actions can nevertheless be
  made

48
Game Theory the Brain (cont)
49
Game Theory the Brain (cont)

• Across trials
• Monkeys behave (near?) optimally their behaviors
  track the Nash equilibrium
• LIP neurons do not track the Nash equilibrium
  suggesting that they are, in fact, encoding
  (relative) expected utility
• Play-by-play
• The relative expected value on any given play
  does vary slightly, given the randomness of play
• Positive correlation b/n this fluctuating
  expected value and fluctuations in LIP neurons

50
Neuroeconomics Language

• Skinners Verbal Behavior
• Programs that are more than input/output
• Bayes Nets for utility as well as beliefs
• Minimum description length grammar
• Minimum description length evolution

51
Neuroeconomics Language

• The paradox disappears only if we make a radical
  break with the idea that language always
  functions in one way, always serves the same
  purpose to convey thoughtswhich may be about
  houses, pains, good and evil, or anything else
  you please. (Sec. 304)