Integer Programming Approaches for Automated Planning

1
Integer Programming Approaches for Automated
Planning
Menkes van den BrielDepartment of Industrial
EngineeringArizona State Universitymenkes_at_asu.ed
uhttp//www.public.asu.edu/dbvan1/
2
What is automated planning?

• Ordering problem
• Scheduling is the problem of deciding when to
  execute a set of actions
• NP-complete

• Selection and ordering problem
• Planning is deciding both what actions need to be
  done and when to execute them
• PSPACE-complete

Scheduling
Planning
3
What is automated planning?

• Creating a computer program to produce a plan, a
  sequence of actions that will transform the world
  from some given initial state to a desired goal
  state

1
2
1
2
Initial states0 ? S
Goalg ? S




PlanP ?a1, , an?
Action
Actions are state transformation functions
4
What is automated planning?

• Creating a computer program to produce a plan, a
  sequence of actions that will transform the world
  from some given initial state to a desired goal
  state

Initial states0 ? S
Goalg ? S




PlanP ?a1, , an?
Action
si
sj
Actions are state transformation functions
5
Planning applications

• Autonomous vehicles
• Mars rovers
• Underwater robotics
• Remote agent experiment
• Games
• Bridge Baron
• General game playing
• Others
• Manufacturing process planning
• Composition of web services
• Cyber Security

6
Planning by integer programming

• Operations research (OR)
• Scheduling problems typically involve solving
  hard optimization problems
• Integer programming (IP), branch-and-bound

• Artificial intelligence (AI)
• Planning problems typically involve solving hard
  feasibility problems
• Constraint satisfaction, satisfiability (SAT), A
  search

Scheduling
Planning
7
Planning by integer programming

• Very little focus on integer programming
  approaches for planning
• Bylander, 1997
• Bockmayr and Dimopoulos, 1998, 1999
• Kautz and Walser, 1999
• Vossen et al., 1999
• Dimopoulos, 2001
• Dimopoulos and Gerevini, 2002

8
Why this lack of interest?

• IP-based approaches simply dont work
• Lplan a linear programming-based heuristic for
  optimal planning was often slower than the other
  algorithms primarily due to the time to evaluate
  the linear programming heuristicBylander,
  1997
• SAT-based approaches are much faster
• SAT-based planners have successfully participated
  in IPC1, IPC2, IPC4, and IPC5
• Traditionally there has been little focus on plan
  quality
• Planning is PSPACE-complete, so finding a
  feasible plan is already hard enough

9
Counter arguments

• IP-based approaches do work
• Optiplan, first IP-based planner to take part in
  the IPC series
• Ranked 2nd in four out of seven domains in IPC4
  in the optimal track for propositional domains
• IP-based approaches can compete with SAT-based
  approaches
• Represent planning as a set of interdependent
  network flow problems
• Generalize the notion of action parallelism
• Shift in focus towards optimal planning
• Applied formulations to partial satisfaction
  planning problems
• Developed a novel framework for optimal planning
• Utilized LP relaxations in deriving quality
  sensitive heuristics

10
Contributions

• IP-based approaches do work
• IP-based approaches can compete with SAT-based
  approaches
• Shift in focus towards optimal planning

• Van den Briel, and Kambhampati. Journal of
  Artificial Intelligence Research, 2005

• Van den Briel, Vossen, and Kambhampati. ICAPS,
  2005
• Van den Briel, Vossen, and Kambhampati. Journal
  of Artificial Intelligence Research, 2008

• Van den Briel, et al. AAAI, 2004
• Do, Benton, van den Briel, and Kambhampati.
  IJCAI, 2007
• J. Benton, van den Briel, and Kambhampati.
  ICAPS, 2007
• Van den Briel, Benton, Kambhampati, and Vossen.
  CP, 2007

11
1. IP approaches do work

• Optiplan
• IP-based planner that extends the state change
  formulation by Vossen et al., 1999

van den Briel, and Kambhampati, 2005
12
Summary of results

• International planning competition (IPC)
• Bi-annual event
• Provides data sets (domains) that are used as
  benchmarks
• IPC4
• 7 competition domains
• 7 participating planners in the optimal track
• Domains
• Pipesworld
• Control the flow of oil derivatives through a
  pipeline network, obeying various constraints
  such as product compatibility and tankage
  restrictions
• Satellite
• Collect image data with a number of satellites
• Philosophers, Optical telegraph
• Involves finding deadlocks in communication
  protocols

13
Summary of results
14
2. IP versus SAT approaches

• Represent planning as a set of interdependent
  network flow problems
• One network flow problem for each state variable
  in the planning domain
• Nodes correspond to the values of the state
  variables, arcs correspond to the value
  transitions
• Generalize the notion of action parallelism
• Reduces the plan length of the solution plan (and
  thus the size of the formulation)

15
Logistics example
1
2
P
T
Truck
Load(P,T,1)Unload(P,T,1)
1
Drive(1,2)
Drive(2,1)
2
Load(P,T, 1)Unload(P,T, 1)
Package
1
Load(P,T, 1)
unload(P,T, 1)
2
Load(P,T, 2)
unload(P,T, 2)
T
States are described by state variables
16
Logistics example
1
2
Prevail
Truck
Load(P,T,1)Unload(P,T,1)
1
Drive(1,2)
Drive(2,1)
2
Load(P,T, 1)Unload(P,T, 1)
Package
1
Load(P,T, 1)
unload(P,T, 1)
Effect
2
Load(P,T, 2)
unload(P,T, 2)
T
Actions are state transformation functions
17
One state change (1SC)

• Network representation
• Logistics example

Prevail
f
f
f
Effect
g
g
g
h
h
h
Plan step
Truck
1
1
2
2
Planning involves considering plans of
increasing length
Package
1
1
2
2
t
t
t 1
18
One state change (1SC)

• Network representation
• Logistics example

Prevail
f
f
f
Effect
g
g
g
h
h
h
Drive(1,2)
Load(P,T, 1)
Unload(P,T, 2)
Truck
1
1
1
1
2
2
2
2
Load(P,T, 1)
Unload(P,T, 2)
-
Package
1
1
1
1
2
2
2
2
t
t
t
t
t 1
t 2
t 3
19
1SC formulation

• Constraints
• State changes (network flow), for all c ? C
  ?g?C ycf,g,t 1f ? I for f ? Dc ?h?C
  ycg,h,t1 ?f?C ycg,h,t for f ? Dc , 1 ? t lt T
  ?f?C ycf,g,T 1 for g ? G
• Effect implications, for all c ? C, 1 ? t ? T
  ?a?A(f,g)?SC(a) xa,t ycf,g,t for f, g ? Dc,
  f ? g xa,t ? ycf,f,t for a ? A, f
  ?PR(a)

20
Summary of results

• Experimental setup
• Domains from IPC2, IPC3
• Comparing 1SC formulation versus SATPLAN04
  (winner of the optimal track IPC4)
• 2.67GHz CPU with 1.0GB memory
• Domains
• Logistics, Driverlog
• Involves driving trucks (and flying airplanes)
  around to deliver packages between locations
• Blocksworld
• Stacking and unstacking towers of blocks
• Zenotravel
• Transporting people around in planes, using
  different modes of movement fast and slow

21
Summary of results
22
2. IP versus SAT approaches

• Represent planning as a set of interdependent
  network flow problems
• One network flow problem for each state variable
  in the planning domain
• Nodes correspond to the values of the state
  variables, arcs correspond to the value
  transitions
• Generalize the notion of action parallelism
• Reduces the plan length of the solution plan (and
  thus the size of the formulation)

23
Generalized one state change (G1SC)

• Network representation
• Example

Prevail
f
f
f
Effect
g
g
g
h
h
h
Load(P,T, 1)Drive(1,2)
Unload(P,T, 2)
Truck
1
1
1
2
2
2
Load(P,T, 1)
Unload(P,T, 2)
Package
1
1
1
2
2
2
t
t
t
t 1
t 2
24
Implied precedences (G1SC)

• Example

A4
A1
A3
A1,A2
A3
A4
A2
Implied precendence graph
25
Implied precedences (G1SC)

• Example
• Ordering (cycle elimination) constraints ensure a
  feasible ordering of the actions

A4
A1
A3
A1,A2
A3
A4
A2
Implied precendence graph
A4
A1
xA1,t xA3,t xA4,t ? 2
26
G1SC formulation

• Constraints
• State changes (network flow), for all c ? C
  ?g?C ycf,g,t 1f ? I for f ? Dc
  ?h?C ycg,h,t1 ?f?C ycg,h,t for f ? Dc, 1 ? t
  ? T ?f?C ycf,g,T 1 for g ? G
• Effect implications, for all c ? C, 1 ? t ? T
• ?a?A(f,f)?SC(a) xa,t ycf,g,t for f, g ? Dc,
  f ? g,
• xa,t ? ycf,f,t ?g?Dcf?g (ycg,f,t ycf,g,t)
  for a ? A, f ?PR(a)
• Ordering (Cycle elimination) constraints
• ? a?V(?) xa,t ? V(?) 1 for all cycles ??G, 1
  ? t ? T

27
Branch-and-cut
START
STOP
Initialize LP
no
Nodes found?
yes
LP solver
Feasible?
no
Node selection
Fathom
yes
Z_lp lt Z?
no
yes
Cut generation
Cuts found?
yes
no
Integer?
no
Branching
yes
28
State change path (PathSC)

• Network representation
• Example

Prevail
f
f
f
Effect
g
g
g
h
h
h
Load(P,T, 1)Drive(1,2)Unload(P,T, 2)
Truck
1
1
2
2
load(P,T, 1)unload(P,T,2)
Package
1
1
2
2
t
t
t 1
29
Summary of results
30
Summary of results
van den Briel, Vossen, and Kambhampati, 2005,
2008
31
3. Shift towards optimal planning

• Applied formulations to partial satisfaction
  planning problems
• Developed a novel framework for optimal planning
• Utilized LP relaxations in deriving quality
  sensitive heuristic search approaches

32
Partial satisfaction planning

• PLAN LENGTH is PSPACE-complete
• Bylander, 1994
• PSP UTILITY COST is PSPACE-complete
• Van den Briel, et al., 2004

Total Satisfaction Problems
PSP UTILITY COST
PSP NET BENEFIT
PSP GOAL LENGTH
PLAN COST
PSP UTILITY
Partial SatisfactionProblems
PLAN LENGTH
PSP GOAL
PLAN EXISTENCE
33
Framework for optimal planning

• For step-based IP formulations optimality is
  restricted to the length of the plan

Plan step
Drive(1,2)
Load(P,T, 1)
Unload(P,T, 2)
Truck
1
1
1
1
2
2
2
2
Load(P,T, 1)
Unload(P,T, 2)
-
Package
1
1
1
1
2
2
2
2
t
t
t
t
t 1
t 2
t 3
34
Framework for optimal planning
1
2
P
T
Truck
Load(P,T,1)Unload(P,T,1)
1
Drive(1,2)
Drive(2,1)
2
Load(P,T, 1)Unload(P,T, 1)
Package
1
Load(P,T, 1)
unload(P,T, 1)
2
Load(P,T, 2)
unload(P,T, 2)
T
35
Action selection formulation

• Variables
• xa ? Z, for a ? A xa is equal to the number of
  times action a is executed
• y?v(c,a) ? Z, for v ? V, a ? A, a ? ?(c)
  y?v(c,a) is equal to the number of times
  transition ?v(c,a) is executed
• Objective function
• MIN ?a?A caxa
• Constraints
• ?a??v(e) y?v(c,a) ?a ??v(e) y?v(c,a) ?
• ?a??v(e) y?v(c,a) xa

No time indicesNo upper bounds
1 if c ? c0,v, c ? g1 if c c0,v, c ?
g0 otherwise
36
Concurrent automata

• Given a set of state variables V v1, , vn
• For each v ? V we define a deterministic
  automaton Gv (Dv, Av, ?v, ?v, c0,v, gv)
• Dv is a finite set of states corresponding to the
  domain of state variable v
• Av is a finite set of actions associated with the
  transitions in Gv
• ?v Dv ? A ? Dv is the transition function
• ?v Dv ? 2A is the active action function
• c0,v ? S is the initial state of state variable v
• gv ? S is a set of goal states of state variable
  v

37
Parallel composition

• The parallel composition of the two automata G1
  and G2 is the automaton G1G2 (D1?D2,
  A1?A2, ?12, ?12, (c0,1, c0,2), g1?g2)
• ?12((c1,c2),a)
• ?12(c1,c2) ?1(c1)??2(c2) ?
  ?1(c1)\A2??2(c2)\A1

(?1(c1,a), ?2(c2,a) if a ? ?1(c1)??2(c2)(?1(c1,a
), c2) if a ? ?1(c1)\A2(c1,?2(c2,a)) if a ?
?2(c2)\A1 undefined otherwise
38
Logistics example
1
2
P
T
Truck
Load(P,T,1)Unload(P,T,1)
1
Drive(1,2)
Drive(2,1)
2
Load(P,T, 1)Unload(P,T, 1)
Package
1
Load(P,T, 1)
unload(P,T, 1)
2
Load(P,T, 2)
unload(P,T, 2)
T
39
Simple logistics example
1
2
P
T
Truck Package
2,1
Drive(2, 1)
Drive(1,2)
1,1
1,2
Load(P, T, 1)
Unload(P, T, 1)
Drive(1, 2)
Drive(2, 1)
1,T
2,2
Unload(P, T, 2)
Drive(2, 1)
Load(P, T, 2)
Drive(1, 2)
2,T
40
Summary of results
Highlighted values equal optimal solution
41
Summary of results
42
Utilize LP in heuristic search
BBOP-LP planner
Benton, van den Briel, and Kambhampati, 2007
43
Summary

• IP-based approaches do work
• Optiplan, first IP-based planner to take part in
  the IPC series
• Ranked 2nd in four out of seven domains in IPC4
  in the optimal track for propositional domains
• IP-based approaches can compete with SAT-based
  approaches
• Represent planning as a set of interdependent
  network flow problems
• Generalize the notion of action parallelism
• Shift in focus towards optimal planning
• Applied formulations to partial satisfaction
  planning problems
• Developed a novel framework for optimal planning
• Utilized LP relaxations in deriving quality
  sensitive heuristics

44
Publications status

• Journal
• M.H.L. van den Briel, and S. Kambhampati.
  Optiplan Unifying IP-based and graph-based
  planning. Journal of Artificial Intelligence
  Research, 24623-635, 2005
• M.H.L van den Briel, T. Vossen, and S.
  Kambhampati. Loosely coupled formulation for
  automated planning An integer programming
  perspective. Journal of Artificial Intelligence
  Research, 31217-257, 2008
• (In progress) M.H.L van den Briel, T. Vossen, S.
  Kambhampati and J. Fowler. Optimal automated
  planning
• Conference
• M.H.L. van den Briel, R. Sanchez, M.B. Do, and
  S. Kambhampati. Effective approaches for partial
  satisfaction (oversubscription) planning. In
  Proceedings of AAAI, pages 562-569, 2004
• M.H.L. van den Briel, T. Vossen, and S.
  Kambhampati. Reviving integer programming
  approaches for AI planning A branch-and-cut
  framework. In Proceedings of ICAPS, pages
  161-170, 2005
• M.B. Do, J. Benton, M.H.L. van den Briel, and S.
  Kambhampati. Planning with goal utility
  dependencies. In Proceedings of IJCAI, pages
  1872-1878, 2007
• J. Benton, M.H.L. van den Briel, and S.
  Kambhampati. A hybrid linear programming and
  relaxed plan heuristic for partial satisfaction
  planning problems. In Proceedings of ICAPS, pages
  24-41, 2007
• M.H.L. van den Briel, J. Benton, S. Kambhampati,
  and T. Vossen. An LP-based heuristic for optimal
  planning. In Proceedings of CP, pages 651-665,
  2007

Cited by 6
Cited by 31
Cited by 15
Cited by 3
Cited by 4
Cited by 3
45
Publications status

• Workshop and posters
• M.H.L. van den Briel, R. Sanchez, and S.
  Kambhampati. Over-Subscription in Planning a
  Partial Satisfaction Problem. In Proceedings of
  ICAPS Workshop on Integrating Planning into
  Scheduling, 2005
• M.H.L. van den Briel,. Kambhampati, and T.
  Vossen. Planning with numerical state variables
  through mixed integer programming. In Proceedings
  of ICAPS Poster Session, pages 5-8, 2005
• M.H.L. van den Briel,. Kambhampati, and T.
  Vossen. Planning with preferences and trajectory
  constraints by integer programming. In
  Proceedings of ICAPS Workshop on Preferences and
  Soft Constraints in Planning, pages 19-22, 2006
• J. Benton, M.H.L. van den Briel,. Kambhampati.
  Finding admissible bounds for oversubscription
  planning problems. In Proceedings of ICAPS
  Workshop on Heuristics for Domain-Independent
  Planning Progress, Ideas, Limitations,
  Challenges, 2007
• M.H.L. van den Briel,. Kambhampati, and T.
  Vossen. Fluent merging A general technique to
  improve reachability heuristics and factored
  planning. In Proceedings of ICAPS Workshop on
  Heuristics for Domain-Independent Planning
  Progress, Ideas, Limitations, Challenges, 2007

Cited by 5
Cited by 1
Cited by 1
Citation count by Google Scholar