Learning Vehicular Dynamics, with Application to Modeling Helicopters

1
Learning Vehicular Dynamics, with Application to
Modeling Helicopters
S T A N F O R D

• Pieter Abbeel, Varun Ganapathi, Andrew Y. Ng

• Overview
• Model-based reinforcement learning has been very
  successful.
• State-of-the-art
• Reinforcement learning returns policies that fly
  well in simulation.
• Remaining helicopter failures typically caused
  by inaccurate simulation.
• Key technical challenge Building an accurate
  simulator.
• Our approach
• Encode all constraints known from physics.
  (Gravity, inertia, etc.) Learn only parts of
  model not determined by physics.
• Explicitly learn simulation that is predictive
  at long time-scales.
• Result
• Significantly improved helicopter model.
• First autonomous funnel (aerobatic maneuver)
  using our model.

• RC Helicopters

Bergen Industrial Twin
XCell Tempest

• Conclusion
• Key technical challenge for model-based
  reinforcement learning applied to helicopters
  building an accurate simulator.
• Our approach
• By using acceleration-based approach, we can
  encode all constraints known from physics.
  (Gravity, inertia, etc.) Learn only parts of
  model not determined by physics.
• Explicitly learn simulation that is predictive
  at long time-scales.
• Result
• Significantly improved helicopter model.
• First autonomous funnel (aerobatic maneuver)
  using our model.