Skill Learning in Telerobotics using Hidden Markov Model

1
Skill Learning in Telerobotics using Hidden
Markov Model

• byGary Holness

2
Skill Learning

• Human performance stochastic
• Repeated trials of same task different
• Something about the task
• Uncovering nature of data
• Most likely performance rejecting noise
• Stochastic methods perfect

3
Why HMM?

• Double stochastic process
• - observable process (motion data)
• - hidden process (mental state/intent)
• Parametric model with incremental update
• Observations as symbols
• Unifying framework for perception and action
• Likely human performance from measured activity

4
Software architecture

• Pre-processing to extract observation symbols
• Algorithm on real-data or simulation

5
SM2 configuration

• 7 DOF
• 6 DOF free flying hand controller provide control
  input

6
HMM experiment

• Orbit replacement unit (ORU)
• Nut driver in gripper
• Model action under teleoperator control as HMM
• Position/trajectory in Cartesian space
• Position/trajectory in Joint space
• Velocity/trajectory in Cartesian space

7
HMM Experiment contd

• Observable symbols trajectory
• States subtasks
• Special case on states time index increases
  left-to-right (Bakis model)
• Fewer parameters than ergodic HMM
• 100 trajectories recorded and scored

8
Position trajectory in Cartesian space

• Forward algorithm used for scoring P(O?)
• Trajectory 60,90 better than average
• Score increase w.r.t iteration ? model improvement

9
Position trajectory in joint space

• Iteration 77 best score
• Velocity in Cartesian, iteration 49 is best

10
What was good?

• Use mathematical framework for which many
  statistical tools already exist
• Integrating framework
• Software engineering
• Rigorous make sound statements about
  experiment (not just it works therefore its
  proven)
• Clearly laid out design

11
What was bad?

• Poor initialization Baum-Welch can converge to
  local maxima
• (not problem of experiment)
• Simplification in left-right HMM
• (understandable why they did it)
• Independence assumption among r.v. in
  R-dimensional observation vector
• (joints non-independent)

12
Why do I care?

• Learning in HMM for ergodic case
• Choose right features as observation symbols
• Make use of 80-years of statistical tools
• Beautifully engineered research artifacts
• Framework for skills transfer and re-use
• Transition among HMMs still an HMM
• lends itself to hierarchical descriptions

13
Conclusion
Representation, representation, representation
14
Wake up! Its over.
Thanks for Listening