State-Space Planning

1
State-Space Planning

• Sources
• Ch. 3
• Appendix A
• Slides from Dana Naus lecture

• Dr. Héctor Muñoz-Avila

2
Reminder Some Graph Search Algorithms (I)
20
G (V,E)
G
Z
10
16
Edges set of pairs of vertices (v,v)
9
B
6
F
C
8
24
4
A
6
E
Vertices
D
20

• Breadth-first search Use a queue (FIFO policy)
  to control order of visits
• State-firsts search Use an stack (FILA policy)
  to control order of visits
• Best-first search minimize an objective
  function f() e.g., distance. Djikstra
• Greedy search select best local f() and never
  look back
• Hill-climbing search like greedy search but
  every node is a solution
• Iterative deepening exhaustively explore up to
  depth i

3
State-Space Planning
C
A

• Search is performed in the state space
• State space
• V set of states
• E set of transitions (s, ?(s,a))
• But computed graph is a subset of state space
• Remember the number of states for DWR states?
• Search modes
• Forward
• Backward

B
C
A
B
C
B
A
B
A
C
A
B
C
B
A
B
C
B
C
A
B
A
C
C
A
A
A
B
C
B
C
C
B
A
A
B
C
4
Nondeterministic Search

• The oracle definition given a choice between
  possible transitions, there is a device, called
  the oracle, that always chooses the transition
  that leads to a state satisfying the goals

...

...
Oracle chooses this transition because it leads
to a favorable state
current state
...

• Alternative definition (book) parallel computers

5
Forward Search
take c3

take c2
move r1

6
Soundness and Completeness

• A planning algorithm A can be sound, complete,
  and admissible
• Soundness
• Deterministic Given a problem and A returns P ?
  failure then P is a solution for the problem
• Nondeterministic Every plan returned by A is a
  solution
• Completeness
• Deterministic Given a solvable problem then A
  returns P ? failure
• Nondeterministic Given a solvable problem then A
  returns a solution
• Admissible If there is a measure of optimality,
  and a solvable problem is given, then A returns
  an optimal solution
• Result Forward-search is sound and complete
• Proof

7
Reminder Plans and Solutions

• Plan any sequence of actions ? ?a1, a2, ,
  an? such thateach ai is a ground instance of an
  operator in O
• The plan is a solution for P(O,s0,g) if it is
  executable and achieves g
• i.e., if there are states s0, s1, , sn such that
• ? (s0,a1) s1
• ? (s1,a2) s2
• ? (sn1,an) sn
• sn satisfies g

Example of a deterministic variant of
Forward-search that is not sound?
Dana Nau Lecture slides for Automated
PlanningLicensed under the Creative Commons
Attribution-NonCommercial-ShareAlike License
http//creativecommons.org/licenses/by-nc-sa/2.0/
8
Deterministic Implementations

• Some deterministic implementationsof forward
  search
• breadth-first search
• depth-first search
• best-first search (e.g., A)
• greedy search
• Breadth-first and best-first search are sound and
  complete
• But they usually arent practical because they
  require too much memory
• Memory requirement is exponential in the length
  of the solution
• In practice, more likely to use depth-first
  search or greedy search
• Worst-case memory requirement is linear in the
  length of the solution
• In general, sound but not complete
• But classical planning has only finitely many
  states
• Thus, can make depth-first search complete by
  doing loop-checking

s1
a1
s4
a4
s0
sg
s2
a2
s5

a5
a3
s3
So why they are typically not used?
How can we make it complete?
Dana Nau Lecture slides for Automated
PlanningLicensed under the Creative Commons
Attribution-NonCommercial-ShareAlike License
http//creativecommons.org/licenses/by-nc-sa/2.0/
9
Another domain The Blocks World

• Infinitely wide table, finite number of
  childrens blocks
• Ignore where a block is located on the table
• A block can sit on the table or on another block
• Want to move blocks from one configuration to
  another
• e.g.,
• initial state goal
• Can be expressed as a special case of DWR
• But the usual formulation is simpler

a
d
b
c
c
e
a
b
10
Classical Representation Symbols

• Constant symbols
• The blocks a, b, c, d, e
• Predicates
• ontable(x) - block x is on the table
• on(x,y) - block x is on block y
• clear(x) - block x has nothing on it
• holding(x) - the robot hand is holding block x
• handempty - the robot hand isnt holding anything

d
c
e
a
b
Dana Nau Lecture slides for Automated
PlanningLicensed under the Creative Commons
Attribution-NonCommercial-ShareAlike License
http//creativecommons.org/licenses/by-nc-sa/2.0/
11
Classical Operators
c
a
b
c
a
b
c
a
b
c
b
a
c
a
b
Dana Nau Lecture slides for Automated
PlanningLicensed under the Creative Commons
Attribution-NonCommercial-ShareAlike License
http//creativecommons.org/licenses/by-nc-sa/2.0/
12
Branching Factor of Forward Search
a1
a2
a3

a50
a3
a1
a2
goal
initial state

• Forward search can have a very large branching
  factor
• E.g., many applicable actions that dont progress
  toward goal
• Why this is bad
• Deterministic implementations can waste time
  trying lots of irrelevant actions
• Need a good heuristic function and/or pruning
  procedure
• Chad and Hai are going to explain how to do this

13
Backward Search

• For forward search, we started at the initial
  state and computed state transitions
• new state ?(s,a)
• For backward search, we start at the goal and
  compute inverse state transitions
• new set of subgoals ?1(g,a)
• To define ?-1(g,a), must first define relevance
• An action a is relevant for a goal g if
• a makes at least one of gs literals true
• g ? effects(a) ? ?
• a does not make any of gs literals false
• g ? effects(a) ? and g ? effects(a) ?

How do we write this formally?
Dana Nau Lecture slides for Automated
PlanningLicensed under the Creative Commons
Attribution-NonCommercial-ShareAlike License
http//creativecommons.org/licenses/by-nc-sa/2.0/
14
Inverse State Transitions

• If a is relevant for g, then
• ?1(g,a) (g effects(a)) ? precond(a)
• Otherwise ?1(g,a) is undefined
• Example suppose that
• g on(b1,b2), on(b2,b3)
• a stack(b1,b2)
• What is ?1(g,a)?
• What is ?1(g,a) s ?(s,a) satisfies g?

15
g1
a1
How about using ?1(g,a) instead of ?1(g,a)?
a4
g4
g2
g0
a2
s0
a5
g5
a3
g3
Dana Nau Lecture slides for Automated
PlanningLicensed under the Creative Commons
Attribution-NonCommercial-ShareAlike License
http//creativecommons.org/licenses/by-nc-sa/2.0/
16
Efficiency of Backward Search
a1
a2
a3

a50
a3
a1
a2
goal
initial state
Compared with forward search?

• Backward search can also have a very large
  branching factor
• E.g., many relevant actions that dont regress
  toward the initial state
• As before, deterministic implementations can
  waste lots of time trying all of them

Dana Nau Lecture slides for Automated
PlanningLicensed under the Creative Commons
Attribution-NonCommercial-ShareAlike License
http//creativecommons.org/licenses/by-nc-sa/2.0/
17
Lifting
p(a,a)
foo(a,a)
p(a,b)
foo(x,y) precond p(x,y) effects q(x)
foo(a,b)
p(a,c)
q(a)
foo(a,c)

• Can reduce the branching factor of backward
  search if we partially instantiate the operators
• this is called lifting

foo(a,y)
q(a)
p(a,y)
Dana Nau Lecture slides for Automated
PlanningLicensed under the Creative Commons
Attribution-NonCommercial-ShareAlike License
http//creativecommons.org/licenses/by-nc-sa/2.0/
18
Lifted Backward Search

• More complicated than Backward-search
• Have to keep track of what substitutions were
  performed
• But it has a much smaller branching factor

Dana Nau Lecture slides for Automated
PlanningLicensed under the Creative Commons
Attribution-NonCommercial-ShareAlike License
http//creativecommons.org/licenses/by-nc-sa/2.0/
19
The Search Space is Still Too Large

• Lifted-backward-search generates a smaller search
  space than Backward-search, but it still can be
  quite large
• Suppose a, b, and c are independent, d must
  precede all of them, and d cannot be executed
• Well try all possible orderings of a, b, and c
  before realizing there is no solution
• More about this in Chapter 5 (Plan-Space Planning)

a
b
d
c
b
a
d
b
a
d
goal
b
a
c
d
b
c
d
a
c
b
d
Dana Nau Lecture slides for Automated
PlanningLicensed under the Creative Commons
Attribution-NonCommercial-ShareAlike License
http//creativecommons.org/licenses/by-nc-sa/2.0/
20
STRIPS

• p ? the empty plan
• do a modified backward search from g
• instead of ?-1(s,a), each new set of subgoals is
  just precond(a)
• whenever you find an action thats executable in
  the current state, then go forward on the current
  search path as far as possible, executing actions
  and appending them to p
• repeat until all goals are satisfied

p ?a6, a4? s ?(?(s0,a6),a4)
g6
satisfied in s0
g1
a1
a4
a6
g4
g2
g
a2
g3
a5
g5
a3
a3
g3
current search path
Dana Nau Lecture slides for Automated
PlanningLicensed under the Creative Commons
Attribution-NonCommercial-ShareAlike License
http//creativecommons.org/licenses/by-nc-sa/2.0/
21
The Sussman Anomaly
a
b
c
c
a
b

• Initial state goal
• On this problem, STRIPS cant produce an
  irredundant solution

Dana Nau Lecture slides for Automated
PlanningLicensed under the Creative Commons
Attribution-NonCommercial-ShareAlike License
http//creativecommons.org/licenses/by-nc-sa/2.0/
22
The Register Assignment Problem

• State-variable formulation
• Initial state value(r1)3, value(r2)5,
  value(r3)0
• Goal value(r1)5, value(r2)3
• Operator assign(r,v,r,v)
• precond value(r)v, value(r)v
• effects value(r)v
• STRIPS cannot solve this problem at all

Dana Nau Lecture slides for Automated
PlanningLicensed under the Creative Commons
Attribution-NonCommercial-ShareAlike License
http//creativecommons.org/licenses/by-nc-sa/2.0/
23
How to Fix?

• Several ways
• Do something other than state-space search
• e.g., Chapters 58
• Use forward or backward state-space search, with
  domain-specific knowledge to prune the search
  space
• Can solve both problems quite easily this way
• Example block stacking using forward search

Dana Nau Lecture slides for Automated
PlanningLicensed under the Creative Commons
Attribution-NonCommercial-ShareAlike License
http//creativecommons.org/licenses/by-nc-sa/2.0/
24
Domain-Specific Knowledge

• A blocks-world planning problem P (O,s0,g) is
  solvableif s0 and g satisfy some simple
  consistency conditions
• g should not mention any blocks not mentioned in
  s0
• a block cannot be on two other blocks at once
• etc.
• Can check these in time O(n log n)
• If P is solvable, can easily construct a solution
  of length O(2m), where m is the number of blocks
• Move all blocks to the table, then build up
  stacks from the bottom
• Can do this in time O(n)
• With additional domain-specific knowledge can do
  even better

25
Additional Domain-Specific Knowledge

• A block x needs to be moved if any of the
  following is true
• s contains ontable(x) and g contains on(x,y)
• s contains on(x,y) and g contains ontable(x)
• s contains on(x,y) and g contains on(x,z) for
  some y?z
• s contains on(x,y) and y needs to be moved

Axioms
Dana Nau Lecture slides for Automated
PlanningLicensed under the Creative Commons
Attribution-NonCommercial-ShareAlike License
http//creativecommons.org/licenses/by-nc-sa/2.0/
26
Domain-Specific Algorithm

• loop
• if there is a clear block x such that
• x needs to be moved and
• x can be moved to a place where it wont
  need to be moved
• then move x to that place
• else if there is a clear block x such that
• x needs to be moved
• then move x to the table
• else if the goal is satisfied
• then return the plan
• else return failure
• repeat

Dana Nau Lecture slides for Automated
PlanningLicensed under the Creative Commons
Attribution-NonCommercial-ShareAlike License
http//creativecommons.org/licenses/by-nc-sa/2.0/
27
Easily Solves the Sussman Anomaly

• loop
• if there is a clear block x such that
• x needs to be moved and
• x can be moved to a place where it wont
  need to be moved
• then move x to that place
• else if there is a clear block x such that
• x needs to be moved
• then move x to the table
• else if the goal is satisfied
• then return the plan
• else return failure
• repeat

initial state
goal
a
c
b
b
a
c
28
Properties

• The block-stacking algorithm
• Sound, complete, guaranteed to terminate
• Runs in time O(n3)
• Can be modified to run in time O(n)
• Often finds optimal (shortest) solutions
• But sometimes only near-optimal (Exercise 4.22 in
  the book)
• PLAN LENGTH for the blocks world is NP-complete

Dana Nau Lecture slides for Automated
PlanningLicensed under the Creative Commons
Attribution-NonCommercial-ShareAlike License
http//creativecommons.org/licenses/by-nc-sa/2.0/
29
An Actual Application FEAR.

• F.E.A.R. First Encounter Assault Recon
• Here is a video clip
• Behavior of opposing NPCs is modeled through
  STRIPS operators
• Attack
• Take cover
• Why?

• Otherwise you would need too build an FSM for all
  possible behaviours in advance

30
FSM vs AI Planning
Neither is more powerful than the other one
31
But Planning Gives More Flexibility

• Separates implementation from data --- Orkin

If conditions in the state change making the
current plan unfeasible replan!