Title: An LPBased Heuristic for Optimal Planning
1An LP-Based Heuristic for Optimal Planning
Menkes van den BrielDepartment of Industrial
EngineeringArizona State Universitymenkes_at_asu.ed
u
J. BentonDepartment of Computer ScienceArizona
State Universitybentonj_at_asu.edu
Subbarao KambhampatiDepartment of Computer
ScienceArizona State Universityrao_at_asu.edu
Thomas VossenLeeds School of BusinessUniversity
of Colorado at Bouldervossen_at_colorado.edu
http//rakaposhi.eas.asu.edu/yochan/
2What is automated planning?
loc1
loc2
Initial states0 ? S
Goals ? S
3What is automated planning?
loc1
loc2
Initial states0 ? S
Goals ? S
loc1
loc1
Action
a ?pre, post, prevail?
4What is automated planning?
loc1
loc2
Initial states0 ? S
Goals ? S
PlanP ?a1, , an?
loc1
loc1
Action
a ?pre, post, prevail?
5Motivation
- Why heuristics?
- Heuristic state space search have been very
successful in solving automated planning problems - Why optimal planning?
- Real-world planning applications require optimal
or near-optimal solutions - The difference between a (near) optimal solution
and a feasible solution may be the difference
between winning or losing the interest of an
investor or strategic partner
6LP-based heuristic
Relax the ordering of the actions
Setup an integer programming formulation
Solve the LP-relaxation and use the objective
function value as an admissible distance estimate
Strengthen the formulation by adding valid
inequalites
7Action selection formulation
- Represent the planning problem as a set of
loosely coupled network flow problems - Each state variable defines one network flow
problem - Nodes correspond to the state variable values
- Arcs correspond to state variable transitions
8Simple logistics example
loc1
loc2
DTGTruck1
Load(p1,t1,l1)Unload(p1,t1,l1)
1
Drive(l1,l2)
Drive(l2,l1)
2
Load(p1,t1,l1)Unload(p1,t1,l1)
DTGPackage1
1
Load(p1,t1,l1)
Unload(p1,t1,l1)
2
Load(p1,t1,l2)
Unload(p1,t1,l2)
T
9Action selection formulation
- Variables
- xa ? Z, for a ? A xa is equal to the number of
times action a is executed - Objective function
- MIN ?a?A xa
- Constraints, for all c ? C, f ? Vc
- ?e?Vc(f)a?AcE(e) xa ?e?Vc(f)b?AcE(e) xb ?
- xa ? M ?e?Vc(f)b?AcE(e) xb for all f ? s0c, a
? AcV(f)
No time indicesNo upper bound
1 if f ? s0c, f sc1 if f s0c, f ?
sc0 otherwise
10Simple logistics example
loc1
loc2
DTGTruck1
Load(p1,t1,l1)Unload(p1,t1,l1)
1
Drive(l1,l2)
Drive(l2,l1)
2
Load(p1,t1,l1)Unload(p1,t1,l1)
DTGPackage1
1
Load(p1,t1,l1)
Unload(p1,t1,l1)
2
Load(p1,t1,l2)
Unload(p1,t1,l2)
T
11Simple logistics example
Drive(l2,l1)
Load(p1,t1,l1)
Drive(l1,l2)
Unload(p1,t1,l2)
DTGTruck1
Load(p1,t1,l1)Unload(p1,t1,l1)
1
Feasible plan xDrive(l2,l1) 1xLoad(p1,t1,l1)
1xDrive(l1,l2) 1xUnload(p1,t1,l2) 1
Drive(l1,l2)
Drive(l2,l1)
2
Load(p1,t1,l1)Unload(p1,t1,l1)
DTGPackage1
1
Load(p1,t1,l1)
Unload(p1,t1,l1)
4
2
Load(p1,t1,l2)
Unload(p1,t1,l2)
T
12Simple logistics example
Drive(l2,l1)
Load(p1,t1,l1)
Unload(p1,t1,l2)
DTGTruck1
Load(p1,t1,l1)Unload(p1,t1,l1)
1
LP solution xLoad(p1,t1,l1) 1xUnload(p1,t1,l2)
1xDrive(l2,l1) 1/M
Drive(l1,l2)
Drive(l2,l1)
2
Load(p1,t1,l1)Unload(p1,t1,l1)
DTGPackage1
1
Load(p1,t1,l1)
Unload(p1,t1,l1)
2 1/M
2
Load(p1,t1,l2)
Unload(p1,t1,l2)
T
13Preliminary results
14Preliminary results
15Strengthening techniques
- Composition of state variables (i.e. fluent
merging) - Given the domain transition graph (DTG) of two
state variables c1, c2, the composition of DTGc1
and DTGc2 is the domain transition graph
DTGc1c2 (Vc1c2, Ec1c2) where - Vc1c2 Vc1 ? Vc2
- ((f1,g1),(f2,g2)) ? Ec1c2 if f1,f2 ? Vc1, g1,g2
? Vc2 and there exists an action a ? A such that
one of the following conditions hold - prec1 f1, postc1 f2, and prec2 g1,
postc2 g2 - prec1 f1, postc1 f2, and prevailc2
g1, g1 g2 - prec1 f1, postc1 f2, and g1 g2
The term composition is also used in model
checking to define the parallel composition or
the synchronized product of automata Cassandras
Lafortune, 1999
16Example
- Two DTGs and their composition
f1,,g1
d
f1,g2
f3,g2
c
f1
a
a
c
c
f2,g1
f3,g1
f2
g1
a
b
b
d
b
d
f2,g2
f3
g2
DTGc1
DTGc2
DTGc1 c2
17Example
- Two DTGs and their composition
- Small in-arcs denote the initial state
- Double circles denote the goal
f1,,g1
d
f1,g2
c
f1
a
a
c
c
f2,g1
f3,g1
f2
g1
a
b
b
d
b
d
f2,g2
f3
g2
DTGc1
DTGc2
DTGc1 c2
18Simple logistics example
loc1
loc2
DTGTruck1 Package1
2,1
Drive(l2,l1)
Drive(l1,l2)
1,1
1,2
Load(p1,t1,l1)
Unload(p1,t1,l1)
Drive(l1,l2)
Drive(l2,l1)
1,T
2,2
Unload(p1,t1,l2)
Drive(l2,l1)
Load(p1,t1,l2)
Drive(l1,l2)
2,T
19Simple logistics example
Drive(l2,l1)
Load(p1,t1,l1)
Drive(l1,l2)
Unload(p1,t1,l2)
DTGTruck1 Package1
2,1
LP solution xDrive(l2,l1) 1xLoad(p1,t1,l1)
1xDrive(l1,l2) 1xUnload(p1,t1,l2) 1
Drive(l2,l1)
Drive(l1,l2)
1,1
1,2
Unload(p1,t1,l1)
Drive(l1,l2)
Drive(l2,l1)
1,T
2,2
Unload(p1,t1,l2)
Drive(l2,l1)
4
Load(p1,t1,l2)
Drive(l1,l2)
2,T
20Another example
- Two DTGs and their composition
f1,,g1
f1,g2
f3,g3
f1
g1
f1,g3
f3,g2
f2
g2
f2,g1
f3,g1
f2,g2
f2,g3
f3
g3
DTGc1
DTGc2
DTGc1 c2
21Another example
- Two DTGs and their composition
- Solution to the individual state variables
f1,,g1
f1,g2
f3,g3
f1
g1
f1,g3
f3,g2
a
b
f2
g2
f2,g1
f3,g1
b
a
f2,g2
f2,g3
f3
g3
DTGc1
DTGc2
DTGc1 c2
22Another example
- Two DTGs and their composition
- Solution to the individual state variables
represented in the composed state variable
f1,,g1
f1,g2
f3,g3
a
f1
g1
f1,g3
f3,g2
b
a
b
f2
g2
f2,g1
f3,g1
b
a
f2,g2
f2,g3
f3
g3
DTGc1
DTGc2
DTGc1 c2
23Another example
- Two DTGs and their composition
- Solution to the individual state variables
represented in the composed state variable
f1,,g1
f1,g2
f3,g3
a
f1
g1
f1,g3
f3,g2
b
a
b
f2
g2
f2,g1
f3,g1
b
a
f2,g2
f2,g3
f3
g3
DTGc1
DTGc2
DTGc1 c2
Violates balance of flow constraints
24Another example
- Two DTGs and their composition
- Adding new balance of flow constraints
strengthens the formulation
f1,,g1
c
f1,g2
f3,g3
a
e
f1
g1
f1,g3
f3,g2
b
a
b
c
f2
g2
d
f2,g1
f3,g1
d
b
a
e
f2,g2
f2,g3
f3
g3
DTGc1
DTGc2
DTGc1 c2
25Identifying mergeable fluents
- When should we create a composition of two or
more state variables? - Look at the causal graph
- Look at the actions that introduce dependencies
in the causal graph
Person 1
Person 2
Person 1
Person 2
Airplane 1
Airplane 2
Airplane 1 Fuel1
Airplane 2 Fuel2
Fuel 1
Fuel 2
26Experimental setup
- Objective
- Minimize number of actions
- Domains
- Selected domains from the International Planning
Competition - Logistics
- Freecell
- Driverlog
- Zenotravel
- TPP
- Blocksworld
- Resources
- 2.67Ghz Linux machine
- 1GB memory
- 15 minutes runtime
- CPLEX 10.0
27Experimental setup
- Distance estimates
- LP
- Action selection formulation with strengthening
- LP
- Action selection formulation without
strengthening - Lplan
- Step based integer programming formulation by
Lplan Bylander, 1997 - h
- Optimal relaxed plan when the delete effects are
ignored - hFF
- Inadmissible but efficient relaxed plan heuristic
by FF Hoffmann, and Nebel, 2001 - Optimal
- Optimal distance estimate given by Satplanner
using the opt flag Rintanen, Heljanko, and
Niemela, 2005
28Experimental results
29Experimental results
Distance estimates from the initial state to the
goal (highlighted values equal the optimal
distance)
30Experimental results
- Heuristic calculation time
31Conclusions and future work
- LP-based heuristic that respects delete effects,
but ignores action ordering shows very promising
results - Finds the optimal distance estimate in several
problem instances - Can be used to calculate admissible distance
estimates for various optimization problems in
planning - Ongoing work successfully incorporated our
LP-based heuristic in a search algorithm that
solves oversubscription planning - Interesting directions for future work
- Apply fluent merging more aggressively
- Extend the formulation into a complete planning
system
32LP-based heuristic
Relax the ordering of the actions
Setup an integer programming formulation
Solve the LP-relaxation and use the objective
function value as an admissible distance estimate
Strengthen the formulation by adding valid
inequalites