Towards Machine Learning of Motor Skills

1
Towards Machine Learning of Motor Skills
Jan Peters, Stefan Schaal, Bernhard Schölkopf

• Philip Dyer

2
What does that mean?

• Machine learning
• The ability of a program to improve its
  performance based on previous results
• Motor skills
• Use of purposeful movements

3
Motive

• We can engineer a robot to act correctly for some
  structured environments
• What about uncontrived environments?
• Some level of autonomous learning must be granted
  to the robot

4
Goal

• Find the foundations for a general framework for
  representing, learning, and executing motor
  skills for robotics

5
Building blocks

• Appropriate representations for movements
• Applicable learning algorithms
• Transformation allowing the execution of the
  kinematic policies in the respective task space

6
Representation

• Each primitive movement in a library is
  represented by nonlinear dynamic systems

7
Learning

• Adapt ?i parameters to learn basic skills
• Sequencing and parallelization of primitives is
  future work
• Generic machine learning techniques do not scale
  well to higher dimensions

8
Learning cont

• Two phases
• 1 Reproduce the skill (supervised learning)
• 2 Improve the skill by trial-and-error
  (reinforcement learning)

9
Execution

• Mechanical system represented by
• can be forced to execute each motor
• primitive
• Policy learning approach replaces control law

10
General concept
11
New learning algorithms

• For long-term reward optimization -
• Natural Actor-Critic
• For immediate improvement -
• Reward-weighted regression

12
Operational space control
13
Motor primitive improvement
14
Questions