Robust Combination of Local Controllers

1
Robust Combination of Local Controllers

• Carlos Guestrin
• Dirk Ormoneit
• Stanford University

2
Planning

• Planning is central in real-world systems
• However, planning is hard
• Motion planning is PSPACE-hard Reif 79
• State and Action spaces are often continuous
• Uncertainty is ubiquitous
• Imprecise actuators
• Noisy sensors.

3
Global versus Local Controllers

• Designing a global controller is hard, but
• Many real-world domains allow us to design good
  local controllers with no global guarantees

How can we combine local controllers to obtain a
global solution ?
4
Combining Local Controllers

• Randomized algorithm
• Nonparametric combination of local controllers
• Generalizes probabilistic roadmaps Hsu et
  al.99
• stochastic domains
• Discounted MDPs
• Theoretical analysis
• Characterizing local goodness of controllers
• polynomial number of milestones is sufficient.

5
Motion Planning Case
Path

• Deterministic motion planning
• Given some start and goal configurations, find a
  collision free path
• Stochastic motion planning
• Given some start and goal configurations, find a
  high probability of success path.

6
Nonparametric Combination of Local Controllers
i
j
Use simulation to estimate quality of local
controllers
Quality prob. controller reaches neighbor
without collisions
7
Nonparametric Combination of Local Controllers
i
pij
j
8
Finding a high success probability path

• Sample milestones uniformly at random
• X1, , XN-1
• Set start as X0 and goal as XN
• Simulation to estimate local connectivity
• Estimate pij for j in the K nearest neigbors of
  i
• Shortest path algorithm to find most probable
  path from X0 to XN
• Edge weights become log pij .

9
Example Maximum Success Probability Path
10
Example Maximum Success Probability Path
11
What About Costs ?

• MDPs find path with lowest expected cost
• Implicit trade-off cost of hitting obstacles and
  reward for goal
• In Robotics, a successful path often more
  important than a short path
• Robotic museum guide
• Manufacturing
• Thus, we make the trade-off explicit
• What is the lowest cost path with success
  probability of at least pmin ?

12
Restricted Shortest Path

• Lowest cost path with success prob. at least
  pmin
• Restricted shortest path problem
• NP-hard, however, FPAS algorithms Hassin 92
• Dynamic programming algorithm
• Discretize pmin,1 into S1 values
• q(s) (pmin)s/S, s 0, , S
• V(s,xi) minimum cost-to-go starting at xi,
  reaching goal with success probability at least
  q(s).

13
ExamplesRestricted Shortest Paths
14
ExamplesRestricted Shortest Paths
Success prob. 0.51 Path length 1.08
Success prob. 0.99 Path length 1.75
15
Theoretical AnalysisCharacterizing quality of
local controllers

• Probabilistic roadmaps (PRMs) Hsu et al. 99
• Deterministic motion planning
• Characterize space as (?,?,?)-good
• Bound number of milestones
• Extension to stochastic domains
• Characterize space and controller as
  (?,?,?,p)-good.

RX points reachable using controller from X
with probability of success ? p
X
RX
Space is (?,p)-good if Volume(RX) ? ? .
Volume(free space)
16
Theorem

• For any ?gt0, a roadmap with N2?8ln(8/???)/??3/?
  ?2 milestones, with probability at least 1-?,
  will contain a path between any two milestones in
  the same connected component and this path will
  have success probability of at least p 3/?1.

In words

• Complete with probability at least 1-?
• Number of milestones poly(ln(1/?), 1/?, 1/?,
  1/?)
• Final path has success probability of at least p
  3/?1.

17
Related Work

• Macro actions in discrete discounted MDPs
• Hauskrecht et al. 1998, Parr 1998
• Probabilistic Roadmaps (PRMs) for deterministic
  motion planning
• Hsu et al. 1999
• Continuous state, discrete actions discounted
  MDPs
• Rust 1997.

18
Centralized Control of Two Holonomic Robots
19
Centralized Control of Two Holonomic Robots
Success prob. 0.54 Total path length 2.79
20
5 dof Robot Arm
Success prob. 0.95 Path length 10.07
Success prob. 0.60 Path length 7.81
21
7 dof Snake
Shortest
Most Success Probaility
Success prob. 0.96 Path length 27.0
Success prob. 0.11 Path length 15.4
22
Conclusions

• Algorithm for planning in stochastic domains with
  continuous state and action spaces
• Nonparametric combination of local controllers
• Motion planning
• Theoretical analysis quantifies local quality of
  controllers
• Proposed alternative objective function
• Qualitative and quantitative properties
  demonstrated
• Also applicable for discounted MDPs
• Describe methods for robustly combining local
  controllers.

http//robotics.stanford.edu/guestrin/Research/Ro
bustLocalControl/