Bayesian goal inference in action observation by cooperating agents

1
Bayesian goal inference in action observation by
co-operating agents
Raymond H. Cuijpers Project Joint-Action
Science and Technology (JAST) Nijmegen Institute
of Cognition and Information Radboud University
Nijmegen The Netherlands
2
Outline

• About Joint Action
• The problem of action observation
• Simulation theory
• Goal inference
• Ingredients functional model
• Functional model of goal inference
• Scenario
• Architecture
• Simulation results
• Conclusions

3
Joint action
About Joint Action

• Multiple levels of co-ordination
• Kinetic force, timing
• Kinematic speed, trajectory
• Action level what to do?
• Goal level for what purpose?
• Reasoning how to achieve destination?
• Actions of co-actors typically differ
• Action observation
• Anticipation of behaviour of co-actor
• Common (ultimate) goal
• Action sequences
• (immediate) action goal inference

4
The problem of action observation
5
How can we infer the observed action?
The problem of action observation

• Simulation theory
• use own motor system to simulate actions of other
• Examples
• Motor control theory
• Forward modelling Predict consequences of
  actions
• Action observation predict observed action from
  action repertoire
• Robotics
• Direct mapping of observed joint angles on those
  of own action repertoire
• Problem
• Requires similar effectors and kinematics
• Perception depends on viewpoint

6
Evidence for simulation theory
The problem of action observation
Mirror Neurons Rizzolatti, Fadiga, Gallese,
Fogassi (1996). Mirror neurons fire
both during observation and execution of similar
actions
Ideomotor compatibility Brass, Bekkering,
Wohlschlaeger, Prinz (2000). Lift finger
indicated by symbol Response is faster
when performing congruent actions

• Shared resources for performing and observing
  actions

• Action system is used in action observation

7
How can we recognise a dog catching a frisbee?
The problem of action observation

• Different body
• Observed effector differs from own effector
  (mouth vs. hand)
• Different kinematics
• Direct mapping of joint angles is impossible
• Forward modelling is impossible

Inference must occur at more abstract level goal
inference
8
Evidence for goal inference
The problem of action observation

• Imitation of 14-months infants
• Gerely, Bekkering, Kiraly (2002). Nature.
• Infants imitate with hands when the actors hands
  were occupied

Hands occupied
Hands free
Imitate using hands
Imitate using head

• Imitate action goals rather than the effector

9
Evidence for goal inference
The problem of action observation

• Mirror neurons
• Fogassi, Ferrari, Gesierich, Rozzi, Chersi
  Rizzolatti (2005). Science 308662-667
• Firing rate during grasping dependson
  subsequent movement
• Activity is selectively tuned to the action goal
  (destination of food)

10
Ingredients for functional model
The problem of action observation

• Viewpoint invariance
• Use viewpoint independent measures (distance,
  colour)
• Infer action goals (intended state change of the
  world)
• make decision at goal level
• consistent with final goal state of a sequence of
  acts
• Use your own action system for observation
• Use own action repertoire
• Use own preferences
• Use own task knowledge

assume common
11
Functional model of goal inference during action
observation
Cuijpers RH, Van Schie HT, Koppen M, Erlhagen W
and Bekkering H (2006) Goals and means in action
observation a computational approach. Neural
Networks 19311-322.
12
Model of action goal inference
Two agents co-operatively build a model from
Baufix building blocks
?
Initial state
Final goal state

• Sequence of primitive motor acts (screw nut, put
  bolt through hole)
• Observable current state and final goal state
  (final construction)
• Shared task knowledge (action repertoire, action
  goals)
• Not shared Action sequence, viewpoint and
  personal preferences

13
Model of action goal inference
Model of action goal inference (Cuijpers et al.,
2006)
Belief that goal is red-bolt-screwed-in-green-nut
Decision
marginalisation rule
Actor
Observer
Belief that action is to screw red bolt in green
nut
marginalisation rule
Belief that hand moves to red bolt
Bayes rule
Likelihood that hand moves to red bolt
Observation
14
Two fundamental processes
Model of action goal inference

• Turn evidence into beliefs (Bayes rule)
• Belief propagation (marginalisation rule)

Pr( red bolt observ. ) Pr( observ. if target
is red bolt ) x Pr( red bolt )
Posterior belief Evidence
Personal preference
Pr(screw red bolt in green nut) Pr(
screw red bolt in green nut if target is n ) x
Pr( target is n )
S
n
Action level Knowledge own action repertoire
Component level
15
Viewpoint invariance
Model of action goal inference

• Observations depend on viewpoint invariant
  measures
• Distance between effector and target
• Rate of distance change

16
Use your own action system
Model of action goal inference

• Belief propagation uses task knowledge
• Components required for each action alternative
  p(cnAk)
• Action goal associated to each action alternative
  p(i?jAk)
• Use personal preferences (priors)
• component preferences p(cn)
• Action preferences p(Ak)
• Action goal preferences for a given final goal
  state p(i?jf)
• Execution and observation share resources
• Task knowledge
• Personal preferences

17
Infer action goals rather than means
Model of action goal inference

• Infer action goal beliefs p(i?jot,f)
• Consistent with final goal state f
• Make decision at goal level
• Belief in action goal p(i?jot, f) gt threshold

18
Simulation results
19
Scenario
Simulation results

• Joint Task
• Actor
• Action Goal bolt through slat
• Action Alternative c1c5
• First target c1
• Observer infer goal

20
Belief component cn is the target p(cnot)
Simulation results

• Nearby targets are more likely unless movement
  speed is high
• Beliefs are biased by personal preferences
• c1 correctly identified after 40 of movement
  time (MT)

21
Belief in action alternatives p(Akot,f)
Simulation results

• Only possible actions (task knowledge)
• Only actions consistent with goal state f (task
  knowledge)
• Action alternatives with nearby targets are more
  likely

Impossible!
Inconsistent
22
Belief in action goals p(i?jot,f)
Simulation results

• Inconsistent action goals are suppressed (task
  knowlegde)
• Correct action goal is inferred after 23 of MT
• The correct action goal is inferred before the
  action or the target component

23
Conclusion

• We made a functional model that captures
  behavioural and neurophysiological findings on
  action observation
• Missing knowledge about the co-actor is replaced
  by task knowledge from the observers own action
  repertoire
• To inference process is driven by the likelihood
  of observed movements and is biased by personal
  preferences
• Action planning is driven by the intended goal
  and by personal preferences
• As a consequence imitation need not involve the
  same effector (imitation)
• Actions are not directly mapped onto the
  observers repertoire.
• Consequently, complementary actions can be as
  fast as imitative actions in a joint action
  context

24
Thank you for your attention!
25
p(cnot) p(otcn) p(cn)
Component belief ? likelihood, preference
p(Akot) Sn p(Akcn) p(cnot) p(i?jot,f) Sk
p(i ? jAk,f) p(Akot)
Action belief ? action knowledge,
component belief Goal belief ? goal knowledge,
action belief
p(i ? jAk,f) p(Aki ? j) p(i ? jf)/
p(Akf) p(Akcn) p(cnAk)p(Ak)/p(cn)