Title: 11th Feb
111th Feb
2h-A
h-B
Pick-A
Pick-B
cl-A
cl-B
he
onT-A
onT-A
onT-B
onT-B
cl-A
cl-A
cl-B
cl-B
he
he
3h-A
on-A-B
St-A-B
on-B-A
h-B
Pick-A
h-A
h-B
Pick-B
cl-A
cl-A
cl-B
cl-B
St-B-A
he
he
onT-A
onT-A
Ptdn-A
onT-A
onT-B
onT-B
onT-B
Ptdn-B
cl-A
cl-A
cl-A
Pick-A
cl-B
cl-B
cl-B
Pick-B
he
he
he
4Using the planning graph to estimate the cost of
single literals
1. We can say that the cost of a single literal
is the index of the first proposition level
in which it appears. --If the literal
does not appear in any of the levels in the
currently expanded planning graph,
then the cost of that literal is
-- l1 if the graph has been expanded to l
levels, but has not yet leveled off
-- Infinity, if the graph has been
expanded
(basically, the literal cannot be achieved from
the current initial state) Examples
h(he) 1 h (On(A,B)) 2 h(he) 0
5Estimating the cost of a set of literals (e.g. a
state in regression search)
Idea 0. Max Heuristic Hmax(p,q,r..)
maxh(p),h(q),. Admissible, but very
weak in practice Idea 2. Sum Heuristic Make
subgoal independence assumption
hind(p,q,r,...) h(p)h(q)h(r)
Much better than set-difference heuristic in
practice. --Ignores ve interactions
h(he,h-A) h(he) h(h-A) 112
But, we can achieve both the literals with just
a single action, Pickup(A). So,
the real cost is 1 --Ignores -ve interactions
h(cl(B),he) 10 1 But, there is
really no plan that can achieve these two
literals in this problem So, the real cost
is infinity!
6Positive Interactions
- We can do a better job of accounting for ve
interactions in two ways - if we define the cost of a set of literals in
terms of the level - hlev(p,q,r) The index of the first level
of the PG where - p,q,r appear together
- so, h(he,h-A) 1
- Compute the length of a relaxed plan to
supporting all the literals in the set S, and use
it as the heuristic () hrelax - hrelax(S) greater than or equal to hlev(S)
- Interestingly, hlev is an admissible
heuristic, even though hind is not! (Prove) How
about hrelax?
7Finding a relaxed plan
- Suppose you want to find a relaxed plan for
supporting literals g1gm on a k-length PG. You
do it this way - Start at kth level. Pick one action for
supporting each gi (the actions dont have to be
distinctone can support more than one goal). Let
the actions chosen be a1aj - Take the union of preconditions of a1aj. Let
these be the set p1pv. - Repeat the steps 1 and 2 for p1pvcontinue until
you reach init prop list. - The plan is called relaxed because you are
assuming that sets of actions can be done
together without negative interactions. - The optimal relaxed plan is the shortest relaxed
plan - Finding optimal relaxed plan is NP-complete
- Greedy strategies can find close-to-shortest
relaxed plan - Length of relaxed plan for supporting S is often
longer than the level of S because the former
counts actions separately, while the later only
considers levels (with potentially more than one
action being present at each level) - Of course, if we say that no more than one action
can be done per level, then relaxed plan length
will not be any higher than level. - But doing this basically involves putting mutex
relations between actions
8Use of PG in Progression vs Regression
Remember the Altimeter metaphor..
- Progression
- Need to compute a PG for each child state
- As many PGs as there are leaf nodes!
- Lot higher cost for heuristic computation
- Can try exploiting overlap between different PGs
- However, the states in progression are
consistent.. - So, handling negative interactions is not that
important - Overall, the PG gives a better guidance
- Regression
- Need to compute PG only once for the given
initial state. - Much lower cost in computing the heuristic
- However states in regression are partial states
and can thus be inconsistent - So, taking negative interactions into account
using mutex is important - Costlier PG construction
- Overall, PGs guidance is not as good unless
higher order mutexes are also taken into account
Historically, the heuristic was first used with
progression planners. Then they used it with
regression planners. Then they found progression
planners do better. Then they found that
combining them is even better.
9Progression
Regression
A PG based heuristic can give two things 1.
Goal-directedness 2. Consistency. Progression
needs 1 more --So can get by without mutex
propagation Regression needs 2 more. --So may
need even higher consistency information
than is provided by normal PG.
10Negative Interactions
- To better account for -ve interactions, we need
to start looking into feasibility of subsets of
literals actually being true together in a
proposition level. - Specifically,in each proposition level, we want
to mark not just which individual literals are
feasible, - but also which pairs, which triples, which
quadruples, and which n-tuples are feasible. (It
is quite possible that two literals are
independently feasible in level k, but not
feasible together in that level) - --The idea then is to say that the
cost of a set of S literals is the index of the
first level of the planning graph, where no
subset of S is marked infeasible -
- --The full scale mark-up is very costly,
and makes the cost of planning graph construction
equal the cost of enumerating the full progres
sion search tree. - -- Since we only want estimates, it is
okay if talk of feasibility of upto k-tuples - -- For the special case of feasibility of
k2 (2-sized subsets), there are some very
efficient marking and propagation procedures. - This is the idea of marking and
propagating mutual exclusion relations.
11- Rule 1. Two actions a1 and a2 are mutex if
- both of the actions are non-noop actions or
- a1 is any action supporting P, and a2 either
needs P, or gives P. - some precondition of a1 is marked mutex with
some precondition of a2
Rule 2. Two propositions P1 and P2 are marked
mutex if all actions supporting P1
are pair-wise mutex with all
actions supporting P2.
12h-A
h-B
Pick-A
Pick-B
cl-A
cl-B
he
onT-A
onT-A
onT-B
onT-B
cl-A
cl-A
cl-B
cl-B
he
he
13h-A
on-A-B
St-A-B
on-B-A
h-B
Pick-A
h-A
h-B
Pick-B
cl-A
cl-A
cl-B
cl-B
St-B-A
he
he
onT-A
onT-A
Ptdn-A
onT-A
onT-B
onT-B
onT-B
Ptdn-B
cl-A
cl-A
cl-A
Pick-A
cl-B
cl-B
cl-B
Pick-B
he
he
he
14- Here is how it goes. We know that at every time
step we are really only going to do one non-no-op
action. So, at the first level either pickup-A,
or pickup-B or pickup-C are done. If one of them
is done, the others cant be. So, we put
red-arrows to signify that each pair of actions
are mutually exclusive. - Now, we can PROPAGATE the mutex relations to the
proposition levels. - Rule 1. Two actions a1 and a2 are mutex if
- both of the actions are non-noop actions or
- a1 is a noop action supporting P, and a2 either
needs P, or gives P. - some precondition of a1 is marked mutex with
some precondition of a2 - By this rule Pick-A is mutex with Pick-B.
Similarly, the noop action he is mutex with
pick-A. - Rule 2. Two propositions P1 and P2 are marked
mutex if all actions supporting P1 are pair-wise
mutex with all actions supporting P2. - By this rule, h-A and h-B are mutex in level 1
since the only action giving h-A is mutex with
the only action giving h-B. - cl(B) and he are mutex in the first level, but
are not mutex in the second level (note that - cl(B) is supported by a noop and stack-a-b
(among others) in level 2. he is supported by
stack-a-b, noop (among others). At least one
action stack-a-b supporting the first is
non-mutex with one actionstack-a-b-- supporting
the second.
15Some observations about the structure of the PG
1. If an action a is present in level l, it will
be present in all subsequent levels. 2.
If a literal p is present in level l, it will be
present in all subsequent levels. 3. If
two literals p,q are not mutex in level l, they
will never be mutex in subsequent levels
--Mutex relations relax monotonically as
we grow PG 1,2,3 imply that a PG can be
represented efficiently in a bi-level
structure One level for propositions and one
level for actions. For each
proposition/action, we just track the first time
instant they got into the PG. For mutex
relations we track the first time instant
they went away. A PG is said to have leveled
off if there are no differences between two
consecutive proposition levels in terms of
propositions or mutex relations. --Even
if you grow it further, no more changes can
occur..
16Level-based heuristics on planning graph with
mutex relations
We now modify the hlev heuristic as follows
hlev(p1, pn) The index of the first level of
the PG where p1, pn appear together
and no pair of them are marked
mutex. (If there is no
such level, then hlev is set to l1 if the PG is
expanded to l levels,
and to infinity, if it has been expanded until it
leveled off)
This heuristic is admissible. With this
heuristic, we have a much better handle on both
ve and -ve interactions. In our example, this
heuristic gives the following reasonable
costs h(he, cl-A) 1 h(cl-B,he) 2
h(he, h-A) infinity (because they
will be marked mutex even in the final level of
the leveled PG)
Works very well in practice
17But Level Doesnt always win over SUM
Solution length/ time in sec.
256MB, 500MHz
1813th Feb
19Qns on PG?
- Consider a set of subgoals p,q,r,s
- If the set appears at level 12 without any pair
being mutex, is there guaranteed to be a plan to
achive p,q,r,s using a 12-step plan? - If p,q appear in level 12 without being mutex,
is there a guaranteed 12-step plan to achieve
p,q? - If p appears in level 12 without being mutex,
is there a guarnateed 12-step plan to achieve p
with ?
PG does approximate reachability analysis
20Massaging good (inadmissible) heuristics from hsum
- Hsum tends to do better sometimes than Hlev (and
vice versa) - Here are some heuristics that do better than both
(can be thought of as adjusting the sum
heuristic) - Combo (add em both! HsumHlev )
- Hsum a negative interaction penalty
- -Degree of interaction in terms of ?
- ?(p,q) lev(p,q) - maxlev(p), lev(q)
- ?(p,q) if p and q are static mutex
- 0 lt ?(p,q) lt if p and q are level specific
mutex - ?(p,q) 0 otherwise (p and q are
non-interacting) - Relaxed plan length a negative interaction
penality - So, start with Hsum and
21Adjusted Sum Heuristic
- Adjust the Sum heuristic to take positive and
negative interactions - HAdjSum2M(S) length(RelaxedPlan(S)) max p,q?S
d(p,q) - Relaxed Plan computed by ignoring all the mutex
relations - Second component is an approximation of a penalty
induced by ignoring the negative interactions - Degree of interaction
- ?(p,q) lev(p,q) - maxlev(p), lev(q)
- ?(p,q) if p and q are static mutex
- 0 lt ?(p,q) lt if p and q are level specific
mutex - ?(p,q) 0 otherwise (p and q are
non-interacting)
Shown to be quite effective
Nguyen Kambhampati, AAAI 2000
22But Level Doesnt always win over SUM
Solution length/ time in sec.
256MB, 500MHz
23A family of Adjusted heuristics
24AltAlt
Uses a hybrid of Level and sum Heuristics
--sacrifices admissibility --uses partial
PG to keep heuristic cost down
25AltAlt Optimizations
- Cost of Computing the Heuristic can be high
- Bi-level Planning Graph representation
- Partial expansion of the PG
- Branching factor can still be quite high
- Use PG to prune unpromising choices
- Select actions in lev(S) vs Levels-off
- (paction pruning actions strategy)
26Bi-Level Planning Graph
- Two Layers of Graph divided into ranks
- A Facts Array
- An Actions Array
- Increases Graph Construction efficiency
- Reduces Storage requirements
- Also quite useful for temporal planning
- problems see TGP
Rank 1
Rank 0
Pointers Level Information
Rank 1
Rank 2
Rank 2
Facts
Actions
27Partial Expansion
- Grow the graph to level k (k lt level-off level)
- Level(S) if k1 instead of , if S is not present
in the graph without mutex
- Can limit the space and time resources expended
on computing the planning graph by trading
heuristic quality
28PGs for reducing actions
- If you just use the action instances at the final
action level of a leveled PG, then you are
guaranteed to preserve completeness - Reason Any action that can be done in a state
that is even possibly reachable from init state
is in that last level - Cuts down branching factor significantly
- Sometimes, you take more risky gambles
- If you are considering the goals p,q,r,s, just
look at the actions that appear in the level
preceding the first level where p,q,r,s appear
for the first time without Mutex.
29Limiting Branching Factor using Planning Graphs
- PACTION strategy Pick actions at lev(S)
- (instead of at the last level)
- Rationale Lev(S) comprises the most significant
actions for achieving state S from the Initial
State
C,D
Reduces BF (albeit incomplete)
Levels-off
Lev(S)
State S
Levels off
30Do PG expansion only upto level(l) where all top
level goals come in without being mutex
AIPS-00 Schedule Domain
Quality
Cost
31Empirical Evaluation Logistics domain
HadjSum2M heuristic
Problems and domains from AIPS-2000 Planning
Competion (AltAlt approx in top four)