Sungwook Yoon, Postdoctoral Research Associate

1
Introduction

• Sungwook Yoon, Postdoctoral Research Associate
• Lecture material and Homework is available on
  course web page
• The lectures are intended to give overview of
  planning research in AI
• Smattering of recent progresses
• Contents
• Classical planning
• Definitions
• Basic algorithmic paradigms
• Markov Decision Processes (MDPs)
• Basic definitions and solution techniques

2
Homework 1

• Due April 25
• Available from Course Web-Page

3
Classical STRIPS Planning
Sungwook Yoon

• Classical planning assumptions
• STRIPS action language
• What is a planning problem?
• complexity of planning
• Planning as Search
• State-space search

Based in part on slides by Alan Fern and Daniel
Weld.
4
AI Planning
Actions
Percepts
World
sole sourceof change vs. other sources
perfect vs. noisy
????
fully observable vs. partially observable
deterministic vs. stochastic
instantaneous vs. durative
5
Classical Planning Assumptions
Actions
Percepts
World
sole sourceof change
perfect
????
deterministic
fully observable
instantaneous
6
Representing States
World states are represented as sets of facts. We
will also refer to facts as propositions.
A
A
B
B
C
C
Closed World Assumption (CWA) Fact not listed
in a state are assumed to be false. Under CWAwe
are assuming the agent has full observability.
7
Representing Goals
Goals are also represented as sets of facts. For
example on(A,B) is a goal in the blocks world.
A goal state is any state that contains all the
goal facts.
A
A
B
B
C
C
State 1 is a goal state for the goal on(A,B) .
State 2 is not a goal state.
8
Representing Action in STRIPS
Stack(A,B)
A
A
B
B
C
C
A STRIPS action definition specifies 1) a
set PRE of preconditions facts 2) a set ADD of
add effect facts 3) a set DEL of delete
effect facts
Stack(x,y) PRE holding(x), clear(y)
ADD on(x,y), handEmpty DEL
holding(x), clear(y)
Stack(A,B) PRE holding(A), clear(B)
ADD on(A,B), handEmpty DEL
holding(A), clear(B)
x?A
x?B
9
Semantics of STRIPS Actions
Stack(A,B)
A
A
B
B
C
C

• A STRIPS action is applicable (or allowed) in a
  state when its preconditions are contained in
  the state.
• Taking an action in a state S results in a new
  state S ? ADD DEL (i.e. add the add effects
  and remove the delete effects)

Stack(A,B) PRE holding(A), clear(B)
ADD on(A,B), handEmpty DEL
holding(A), clear(B)
10
STRIPS Planning Problems
A STRIPS planning problem specifies 1) an
initial state S 2) a goal G 3) a set
of STRIPS actions
Objective find a short action sequence
reaching a goal state, or
report that the goal is unachievable
Example Problem
Solution (Stack(A,B))
A
B
Stack(A,B) PRE holding(A), clear(B)
ADD on(A,B), handEmpty DEL
holding(A), clear(B)
Stack(B,A) PRE holding(B), clear(A)
ADD on(B,A), handEmpty DEL
holding(B), clear(A)
STRIPS Actions
11
Properties of Planners

• A planner is sound if any action sequence it
  returns is a true solution
• A planner is complete outputs an action sequence
  or no solution for any input problem
• A planner is optimal if it always returns the
  shortest possible solution

Is optimality an important requirement? Is it a
reasonable requirement?
12
Complexity of STRIPS Planning
PlanSAT Given a STRIPS planning problem
Output yes if problem is solvable, otherwise
no

• PlanSAT is decidable.
• Why?
• In general PlanSAT is PSPACE-complete! Just
  finding a plan is hard in the worst case.
• even when actions limited to just 2 preconditions
  and 2 effects

Does this mean that we should give up on AI
planning?
NOTE PSPACE is set of all problems that are
decidable in polynomial space.
PSPACE-complete is believed to be harder than
NP-complete
13
Satisficing vs. Optimality

• While just finding a plan is hard in the worst
  case, for many planning domains, finding a plan
  is easy.
• However finding optimal solutions can still be
  hard in those domains.
• For example, optimal planning in the blocks world
  is NP-complete.
• In practice it is often sufficient to find good
  solutions quickly although they may not be
  optimal.
• For example, finding sub-optimal blocks world
  solutions can be done in linear time. How?

14
Propositional Planners

• For clarity we have written propositions such as
  on(A,B) in terms of objects (e.g. A and B) and
  predicates (e.g. on).
• However, the planners we will consider ignore the
  internal structure of propositions such as
  on(A,B).
• Such planners are called propositional planners
  as opposed to first-order or relational planners
• Thus it will make no difference to the planner if
  we replace every occurrence of on(A,B) in a
  problem with prop1 (and so on for other
  propositions)
• It feels wrong to ignore the existence of
  objects. But currently propositional planners are
  the state-of-the-art.

15
STRIPS Action Schemas
For convenience we typically specify problems via
action schemas rather than writing out
individual STRIPS action.
Stack(B,A) PRE holding(B), clear(A)
ADD on(B,A), handEmpty DEL
holding(B), clear(A)
Action Schema (x and y are variables)
Stack(x,y) PRE holding(x), clear(y)
ADD on(x,y), handEmpty DEL
holding(x), clear(y)
. . . .
Stack(A,B) PRE holding(A), clear(B)
ADD on(A,B), handEmpty DEL
holding(A), clear(B)

• Each way of replacing variables with objects from
  the initial state and goal yields a ground
  STRIPS action.
• Given a set of schemas, an initial state, and a
  goal, propositional planners compile schemas into
  actions and then ignore the existence of objects
  thereafter.

16
Planning as Search

• Nearly all classical planners are search
  procedures
• Different planners use different search spaces
• Two examples
• State-space planning
• Each search node represents a state of the world
• Search finds a path through the space (a plan)
• Plan-space planning
• Each search node is a partial plan giving
  constraints on a plan
• Search adds more constraints until we get a
  complete plan
• We will cover classical planners in historical
  order.

17
State-Space Search

• Search Nodes states
• Search Arcs STRIPS actions
• Solution path from the initial state to one
  state that satisfies the goal
• Initial state is fully specified
• There can be many goal states

18
Search Space Blocks World
Search space is finite.
19
Forward-Chaining Search
. . . .
goal
initial state
. . . .

• Breadth-first and best-first search are sound and
  complete
• Very large branching factor can cause search to
  waste time and space trying many irrelevant
  actions
• O(bd) worst-case where b branching factor, d
  depth limit
• Need a good heuristic function and/or pruning
  procedure
• Early AI researchers gave up on forward search.
• But there has been a recent resurgence. More on
  this later in the course.

20
Backward-Chaining Search
goal
initial state
. . . .

• Backward search can focus on more goal relevant
  actions, but still the branch factor is typically
  huge
• Again a good heuristic function and/or pruning
  procedure
• Early AI researchers gave up on forward and
  backward search.
• But there has been recent progress in developing
  general planning heuristics leading to a
  resurgence. More on this later in the course.

21
Total-Order vs. Partial-Order Planning (POP)
?
A
D
C
B
B
D
A
C
There are many possible plans

• move(A, B, TABLE) move(B, TABLE, A) move(C,
  D, TABLE) move(D, TABLE, C)
• move(A, B, TABLE) move(C, D, TABLE) move(D,
  TABLE, C) move(B, TABLE, A)
• move(C, D, TABLE) move(D, TABLE, C) move(A,
  B, TABLE) move(B, TABLE, A)
• ect . . .

• State-space planning techniques produce
  totally-ordered plans, i.e. plans consisting of a
  strict sequence of actions.
• Often, however, there are many possible orderings
  of actions than have equivalent effects.
• However, often many orderings of the actions have
  equivalent effects.

22
Total-Order vs. Partial-Order Planning (POP)
?
A
D
C
B
B
D
A
C
There are many possible plans

• move(A, B, TABLE) move(B, TABLE, A) move(C,
  D, TABLE) move(D, TABLE, C)
• move(A, B, TABLE) move(C, D, TABLE) move(D,
  TABLE, C) move(B, TABLE, A)
• move(C, D, TABLE) move(D, TABLE, C) move(A,
  B, TABLE) move(B, TABLE, A)
• ect . . .

• These plans share some common structure. They are
  all different interleavings of two separate
  plans
• 1) move(A, B, TABLE) move(B, TABLE,
  A)
• 2) move(C, D, TABLE) move(D, TABLE,
  C)
• A partial-order plan is one which specifies only
  the necessary ordering information. One
  partial-order plan may have many total-orderings

23
Total-Order vs. Partial-Order Planning (POP)
24
(Partial-Order) Plan-Space Search

• Search nodes are partial plans
• some plan steps may be missing
• some orderings of actions may not be finalized
• Search arcs are plan refinements
• Solution is a node (not a path) giving the a
  partial order on actions.
• Any linearization of the plan is a total-order
  solution to the problem

It follows the least-commitment principle Do
not add constraints (e.g. action ordering) to a
plan until it becomes necessary to ensure the
correctness of the plan.
25
Representing Partial-Order Plans

• Plan (A, O, L, P), where
• A set of actions in the plan
• O temporal orderings between actions (a lt b)
• L causal links linking actions via a literal
  (used for book keeping)
• P open preconditions that need to be satisfied
• Causal Link
• Action A2 has precondition Q that is
  established in the plan by action A1.

clear(b)
Unstack(C,B)
Putdown(A,B)
26
Threats to a Causal Link

• A causal link (A1, Q, A2) represents the
  assertion that the role of A1 is to establish
  proposition Q for A2
• This tells future search steps to protect Q in
  the interval between A1 and A2

Action B threatens causal link (A1, Q, A2) if 1.
B has Q as a delete effect, and 2. B could come
between A1 and A2, i.e. O ? (A1 lt B lt A2)
is consistent
PutDown(C,B)
What is a threat to the following causal link?
clear(B)
Unstack(C,B) PutDown(A,B)
27
POP example Blocks World
Initial plan includes only START and FINISH
actions.The open preconditions correspond to the
goal.
START
on(C, A)
ontable(A)
ontable(B)
clear(C)
clear(B)
C
B
A
on(A, B)
on(B, C)
A
FINISH
B
C
28
Work on open precondition on(B, C)
and clear(B)
START
on(C, A)
ontable(A)
ontable(B)
clear(C)
clear(B)
C
B
A
preconditions
effects
on(A, B)
on(B, C)
A
FINISH
B
C
29
Work on open precondition on(A, B)
START
on(C, A)
ontable(A)
ontable(B)
clear(C)
clear(B)
C
B
A
clear(A)
clear(B)
ontable(A)
A2 move(A, TABLE, B)
on(A, B)
ontable(A)
clear(B)
on(A, B)
on(B, C)
A
FINISH
B
C
30
Protect causal links
START
on(C, A)
ontable(A)
ontable(B)
clear(C)
clear(B)
C
B
A
clear(A)
clear(B)
ontable(A)
A2 move(A, TABLE, B)
on(A, B)
ontable(A)
clear(B)
on(A, B)
on(B, C)
A
FINISH
B
C
31
Work on open precondition clear(A)
START
on(C, A)
ontable(A)
ontable(B)
clear(C)
clear(B)
C
B
A
clear(A)
clear(B)
ontable(A)
A2 move(A, TABLE, B)
on(A, B)
ontable(A)
clear(B)
on(A, B)
on(B, C)
A
FINISH
B
C
32
Final Plan
All preconditions are supported and plan is
consistent.So this partial-order plan is a
solution.
START
on(C, A)
ontable(A)
ontable(B)
clear(C)
clear(B)
C
B
A
clear(C)
on(C, A)
A3 move(C, A, Table)
on(C, A)
ontable(C)
clear(A)
clear(B)
clear(C)
ontable(B)
A1 move(B, TABLE, C)
clear(A)
clear(B)
ontable(A)
on(B, C)
ontable(B)
clear(C)
A2 move(A, TABLE, B)
on(A, B)
ontable(A)
clear(B)
on(A, B)
on(B, C)
A
FINISH
B
C