Title: 27: Knowledgebased Planning
12/7 Knowledge-based Planning
2Life must be lived forward, but can only be
understood backward.--Kierkegaard
Lp relaxation
3Should we bother with Proof-based encodings?
- Clearly Graphplan-based encodings are better than
backward-proof based encodings - Because Graphplan does Mutex propagation
- And mutex propagation is harder to do on direct
SAT - And yet, it is useful to know how to set up
encodings directlysince if we dont know how to
do directed consistency enforcement, then we at
least know how to solve the problem - Particularly useful for non-classical problems,
where we havent yet found really effective mutex
propagation procedures. - If neural networks are often the second best way
of solving any problem, you can say that direct
SAT(or other) encodings are the second best way
of solving planning problems - Which is better than no way at all
4Connections to Refinement Planning Framework
5Disjunctive Planning
- Idea Consider Partial plans with disjunctive
step, ordering, and auxiliary constraints - Motivation Provides a lifted search space,
avoids re-generating the same failures multiple
times (also, rich connections to combinatorial
problems) - The solution check now involves searching
through the space of all (exponentially many)
minimal candidates of the plan to see if any of
them is a solution - Issues
- Refining disjunctive plans
- Graphplan (Blum Furst, 95)
- Solution extraction in disjunctive plans
- Direct combinatorial search
- Compilation to CSP/SAT/ILP
6Disjunctive Representations
--Allow disjunctive step, ordering and
auxiliary constraints in partial plans
lt 1,In(A), ? gt V lt 1 ,In(B), ? gt
In(A)
?
0
1 Load(A)
Load(A)
In(x)_at_ ?
?
0
1 or
In(B)
Load(B)
?
0
1 Load(B)
In(x)_at_ ?
At(A,E)_at_1 V At(B,E)_at_1
7Refining Disjunctive Plans (1)
Indirect unioned approach
Maximum pruning power - Exponential refinement
cost INFEASIBLE
8Refining Disjunctive plans (2)
Direct naive approach Put all actions at all
levels --Proposition list contains all
conditions
Trivial to construct - Loss of pruning power
(progressivity) gt too many (minimal)
candidates gt Costlier solution extraction
1 Load(A,E)
or
2 Load(B,E)
or
3 Fly(R)
or
4 Unload(A,E)
USED in SATPLAN
or
5 Unload(B,E)
6 Unload(A,M)
7 Unload(B,M)
8 load(A,M)
9 load(B,M)
9Refining Disjunctive plans (2)
Direct naive approach Put all actions at all
levels --Proposition list contains all
conditions
Trivial to construct - Loss of pruning power
(progressivity) gt too many (minimal)
candidates gt Costlier solution extraction
USED in SATPLAN
10Refining Disjunctive plans (3)
Enforce partial 1-consistency Proposition
list avoids unsupported conditions
Polynomial refinement time -- Loss of pruning
power (progressivity)
1 Load(A,E)
or
2 Load(B,E)
or
3 Fly(R)
or
4 Unload(A,E)
or
5 Unload(B,E)
6 Unload(A,M)
7 Unload(B,M)
8 load(A,M)
9 load(B,M)
11Refining Disjunctive plans (4)
Enforce (partial) 2-consistency Proposition
list maintains interactions between pairs
of conditions
Polynomial refinement time higher pruning
power Better balance between refinement cost
and solution extraction cost
1 Load(A,E)
or
2 Load(B,E)
or
3 Fly(R)
or
4 Unload(A,E)
or
5 Unload(B,E)
6 Unload(A,M)
Can do higher level consistency, but it is rarely
worth the cost
7 Unload(B,M)
8 load(A,M)
9 load(B,M)
12CUSTOMIZING PLANNERS WITH DOMAIN SPECIFIC
KNOWLEDGE
- User-assisted customization
- Planning by Cheating??
- (accept domain-specific knowledge as input)
- 2. Automated customization
- (learn regularities of the domain through
analysis or - experience)
13Digression Domain-Independent vs. Domain
Specific vs. Domain Customized
Review
- Domain independent planners only expect as input
the description of the actions (in terms of their
preconditions and effects), and the description
of the goals to be achieved - Domain dependent planners make use of additional
knowledge beyond action and goal specification - Domain dependent planners may either be stand
alone programs written specifically for that
domain OR domain independent planners customized
to a specific domain - In the case of domain-customized planners, the
additional knowledge they exploit can come in
many varieties (declarative control rules or
procedural directives on which search choices to
try and in what order) - The additional knowledge can either be input
manually or in some cases, be learned
automatically
Unless noted otherwise, we will be talking about
domain-independent planning
14Improving Performancethrough Customization
- Biasing the search with control knowledge
acquired from experts - Non-primitive actions and reduction schemas
- Automated synthesis of customized planners
- Combine formal theory of refinement planning and
domain-specific control knowledge - Use of learning techniques
- Search control rule learning
- Plan reuse
15User-Assisted Customization(using
domain-specific Knowledge)
- Domain independent planners tend to miss the
regularities in the domain - Domain specific planners have to be built from
scratch for every domain - An Any-Expertise Solution Try adding domain
specific control knowledge to the
domain-independent planners
16Domain KnowledgePrescriptive or Suggestive?
- One big question is whether the additional domain
knowledge given to the planner - suggestive? (soft constraint on the search and
solution) - OR
- Prescriptiive? (hard constraint on the search and
solution) - At the outset, it would seem like either way
should be fine. However, handling
soft-constraints correctly is harder than
handling hard-constraints.. - So, most knowledge-based planners use the domain
knowledge as prescriptive
17Task Decomposition (HTN) Planning
- The OLDEST approach for providing domain-specific
knowledge - Most of the fielded applications use HTN planning
- Domain model contains non-primitive actions, and
schemas for reducing them - Reduction schemas are given by the designer
- Can be seen as encoding user-intent
- Popularity of HTN approaches a testament of ease
with which these schemas are available? - Two notions of completeness
- Schema completeness
- (Partial Hierarchicalization)
- Planner completeness
18Modeling Reduction Schemas
GobyBus(S,D)
In(B)
t1 Getin(B,S)
Hv-Tkt
t2 BuyTickt(B)
t3 Getout(B,D)
Hv-Money
At(D)
19Stuff to add..
- Domain knowledge as prescriptive vs. suggestive
- Recipes as HTN schemas whose correctness is not
known - The fact that soundness of HTN schemas can be
checked but not completeness even with full
domain model at the prec/effect level - Actually, we define soundness in terms of User
liking the solution This argument makes sense
only if soundness is in terms of executability..
20(No Transcript)
21(No Transcript)
22(No Transcript)
23(Thoroughly) Mixed Motivations for HTN Planning
To allow for use of Abstraction/Hierarchy in
modeling
To provide a grammar For desirable solutions
This is about the only thing That we can say
about HTNs If we dont put any restrictions On
the schemas. However, the question of
Validation of schemas then is External to the
planner since Schemas are supposedly Capturing
what kinds of solutions Users like. It is not
very clear that HTNs are the best way to
express preferences on solutions anyway
Abstraction used mostly for speedup
To model Partial domain knowledge (when the
domain Writer doesnt have Full domain model)
Unless we put strong restrictions on HTN
schemas it is hard to connect any notion of
abstractionsuch as Downward refinability-- to
HTN schemas. (Since HTNs can allow recursive
subgoals e.g. Empty Truck reduced to get a
package out, Empty Truck, conditions may not
have any real connection to abstraction
HTNs provde a natural Substrate for this, But
there has been Little formal use of HTNs this way
24A concretization is a task network produced by
repeatedly reducing a non-primitive task
until all tasks are primitive
25Modeling Action Reduction
Affinity between reduction schemas and plan-space
planning
26Dual views of HTN planning
- Capturing hierarchical structure of the domain
- Motivates top-down planning
- Start with abstract plans, and reduce them
- Many technical headaches
- Respecting user-intent, maintaining systematicity
and minimality - Kambhampati et. al. AAAI-98
- Phantomization, filters, promiscuity,
downward-unlinearizability..
- Capturing expert advice about desirable solutions
- Motivates bottom-up planning
- Ensure that each partial plan being considered is
legal with respect to the reduction schemas - Directly usable with disjunctive planning
approaches - Connection to efficiency is not obvious
Relative advantages are still unclear...
Barrett, 97
272/12
28(No Transcript)
29Changes to Plan-Space Refinement(in the presence
of non-primitive tasks)
30(No Transcript)
31HTN vs. T-STN
- Naus book talks about a special version of HTN
called T-STN (Total-ordering STN) - T-STN schemas are HTN schemas where all the
ordering relations between tasks are constrained
to be contiguity constraints - This means that the concretizations of different
tasks cannot be interleaved - Bug or feature?
- From the planners point of view, it can be a
feature Most of the difficulties in HTN planning
come from the need to do plan-space refinement in
the presence of non-primitive tasks
(phantomization, conflict resolution etc..)
eliminating that will make HTN planning a
no-brainer - BUT from the domain-writers point of view it is
clearly a BUG - Would have to write many more schemas
- Same is true of Yangs unique main sub-action
thing (although it is less drastic than T-STN) - (few cooking recipes are given with contiguity
orderingswhich means that you cant do anything
else while following the recipe) - The only HTN planners that use STN schemas are
those from Naus group ? - SIPE and O-Plan use schemas with precedence
orderings
32(No Transcript)
33(No Transcript)
34HTN vs. STRIPS planningCompeting or
Complementary?
- Much of the discussion till now has been done
with the assumption that HTN is just a way of
doing STRIPS planning
- HTN can also be seen as solving a different
problem - Can develop plans at various levels of
abstraction - Can model domains with partial domain knowledge
- What you consider primitive action may be
changeable - What you thought was a correct plan may not be..
359/6
36Full procedural control The SHOP way
Nau et. al., 99
Shop provides a high-level programming language
in which the user can code his/her domain
specific planner -- Similarities to HTN
planning Uses T-STN schemas -- Order
of use of reduction schemas can be
specified The SHOP engine can be seen as
an interpreter for this language
Blurs the domain-specific/domain-independent
divide How often does one have this level of
knowledge about a domain?
37Non-HTN Declarative Guidance
Kautz Selman, AIPS-98
Invariants A truck is at only one location
at(truck, loc1, I) loc1 ! loc2 gt at(truck,
loc2, I) Optimality Do not return a package to
a location at(pkg, loc, I)
at(pkg,loc,I1) IltJ gt at(pkg,loc,j) Simplify
ing Once a truck is loaded, it should
immediately move in(pkg,truck,I)
in(pkg,truck,I1) at(truck, loc, I1) gt
at(truck, loc, I2) Once again,
additional clauses first increase the encoding
size but make them easier to solve after
simplification (unit-propagation etc).
38Case Study SATPLAN with domain specific knowledge
- Instantiate the domain specific rules as well,
and then add them to the encoding. - Solve the full encoding
- Wont the encoding size increase with domain
spefic rules? - Yes, but the new rules support a lot of
simplification so that after simplification, the
encoding size actually reduces!! - Example in Blocks world (from Kautz Selman,
AIPS-99) - BW-c
Original 4258 vars (10 increase) / 30986
clauses (300 increase)
Final (after simplification)
3526 vars 22535 clauses
39SAT encodings of HTN planning
Mali Kambampati, AIPS-98
- Abstract actions can be seen as disjunctive
constraints - K-step encoding has each of the steps mapped to a
disjunction of the non-primitive tasks - If a step s is mapped to a task N, then one of
the reductions of N must hold (The heart of
encoding setup) - The normal constraints of primitive
action-based encoding - Causal encodings seem to be a natural fit (given
the causal dependencies encoded in reduction
schemas)
Solutions for action based encodings
HTN-compliant Solutions
40Solving HTN Encodings
Puzzle How can increasing encoding sizes lead to
efficient planning? Abstract actions and their
reductions put restrictions on the amount of
step-action disjunction at the primitive level.
--Reduction in step-action disjunction
propagates e.g. Fewer causal-link
variables, Fewer exclusion clauses Savings
wont hold if each non-primitive task has MANY
reductions
Kambhampati Mali, AIPS-98
41Rules on desirable State Sequences TLPlan
approach
TLPlan Bacchus Kabanza, 95/98 controls a
forward state-space planner Rules are
written on state sequences using the linear
temporal logic (LTL) LTL is an extension of prop
logic with temporal modalities U until
always O next
ltgt eventually Example If
you achieve on(B,A), then preserve it until
On(C,B) is achieved ( on(B,A) gt
on(B,A) U on(C,B) )
42TLPLAN Rules can get quite boroque
Good towers are those that do not violate any
goal conditions
Keep growing good towers, and avoid bad towers
How Obvious are these rules?
The heart of TLPlan is the ability to
incrementally and effectively evaluate the truth
of LTL formulas.
43Control rules for choice points
- UCPOP developed a language in which user can
write control rules that tell the planner what to
do when it faces a specific sort of choice (e.g.
between open conditions establishers for open
conditions etc.). - HSTSa metric temporal planner that NASA used in
its DeepSpace mission used similar control rule
language. - The user needs to know the innards of the planner
to write these rules - The rules are hard to debug
- Learning such rules automatically is a way out
44Folding the Control Knowledge into the planner
CLAY approach
Srivastava Kambhampati, JAIR 97
- Control knowledge similar to TLPlans
- Knowledge is folded using KIDS semi-automated
software synthesis tool into a generic refinement
planning template - Use of program optimizations such as
- Finite differencing
- Context-dependent independent simplification
- Empirical results demonstrate that folding can be
better than interpreting rules
Caveat Current automated software synthesis
tools have a very steep learning curve
45Many User-Customizable Planners
- Conjunctive planners
- HTN planners
- SIPE Wilkins, 85-
- NONLIN/O-Plan Tate et. al., 77-
- NOAH Sacerdoti, 75
- Also SHOP (Nau et. al., IJCAI-99)
- State-space planners
- TLPlan Bacchus Kabanza, 95 99
- TALPlan Kvarnstrom Doherty, 2000
- Customization frameworks
- CLAY Srivastava Kambhampati, 97
- Disjunctive planners
- HTN SAT Mali Kambhampati, 98
- SATPLANDom Kautz Selman, 98
How do they relate? What are the tradeoffs?
46With right domain knowledge any level of
performance can be achieved...
- HTN-SAT, SATPLANDOM beat SATPLAN
- Expect reduction schemas, declarative knowledge
about inoptimal plans - TLPLAN beats SATPLAN, GRAPHPLAN
- But uses quite detailed domain knowledge
- SHOP beats TLPLAN(but not TALPlan)
- Expects user to write a program for the domain
in its language - Explicit instructions on the order in which
schemas are considered and concatenated
Who gets the credit? -The provider of domain
knowledge? -The planner that is capable of
using it?
47Types of Guidance
- Declarative knowledge about desirable or
undesirable solutions and partial solutions
(SATPLANDOM Cutting Planes) - Declarative knowledge about desirable/undesirable
search paths (TLPlan TALPlan) - A declarative grammar of desirable solutions
(HTN)
(largely) independent of the details of the
specific planner affinities do exist between
specific types of guidance and planners)
Planner specific. Expert needs to understand the
specific details of the planners search space
- Procedural knowledge about how the search for the
solution should be organized (SHOP) - Search control rules for guiding choice points in
the planners search (NASA RAX)
48Where does guidance come from?
- Learned from (planning) experience
- EBL-based search control rule learning
- (UCPOPEBL), PRODIGY
- inductive learning
- Case-based reasoning (DerSNLP)
- Given by humans (often, they are quite willing!)
- As declarative rules (HTN Schemas, Tlplan rules)
- Dont need to know how the planner works..
- Tend to be hard rules rather than soft
preferences - Whether or not a specific form of knowledge can
be exploited by a planner depends on the type of
knowledge and the type of planner - As procedures (SHOP)
- Direct the planners search alternative by
alternative..
how easy is it to write control information?
49Ways of using the Domain Knowledge
- As search control
- HTN schemas, TLPlan rules, SHOP procedures
- Issues of Efficient Matching
- To prune unpromising partial solutions
- HTN schemas, TLPlan rules, SHOP procedures
- Issues of maintaining multiple parses
- As declartative axioms that are used along with
other knowledge - SATPlanDomain specific knowledge
- Cutting Planes (for ILP encodings)
- Issues of domain-knowledge driven simplification
- Folded into the domain-independent algorithm to
generate a new domain-customized planner - CLAY
- Issues of Program synthesis
50Conundrums of user-assisted cutomization
- Which planners are easier to control?
- Conjunctive planners are better if you have
search control knowledge - Forward State Space (according to TLPlan)
- Plan-space planners (according to HTN approaches)
- Disjunctive planners are better if your
knowledge can be posed as additional constraints
on the valid plans - Which SAT encoding?
- HTN knowledge is easier to add on top of causal
encodings - Which approach provides the best language for
expressing domain knowledge for the lay user? - (Mine--no, no, Mine!)
- What type of domain knowledge is easier to
validate? - When does it become cheating/
wishful-thinking - Foolish not to be able to use available knowledge
- Wishful to expect deep procedural knowledge...
51Evaluating KB-planners
- Since the 2nd IPC, there has been a separate
track for knowledge-based planners at the
competition. - Basically only the TLPlan (and its variant
TALPlan) and SHOP took part (in comparison close
to a dozen planners take part in the
domain-independent track) - In 2000, TALPlan won
- In 2002, TLPlan won
- Quite controversial
- When SHOP does better than TLplan on Gizmo
domain, what does this mean? - That SHOP is a better planning algorithm than
TLplan? - That Dana Nau knows Gizmo domain better than
Faheim Bacchus? - That the easily available control knowledge in
Gizmo domain is more naturally representable as
TLplan rules than SHOP schemas? -
52Are we comparing Dana Fahiem or SHOP and
TLPlan?(A Critique of Knowledge-based Planning
Track at ICP)
- Subbarao Kambhampati
- Dept. of Computer Science Engg.
- Arizona State University
- Tempe AZ 85287-5406
53The I am not an anti-dentiteDisclaimers..
- I think KB planning is a swell idea
- I started my career with HTN planning
- I think the KB planning track at IPC is a swell
idea - has done more to increase interest in KBplanning
than the bi-annual polemics and laments about
lack of interest in Knowledge-based planning - I think Fahiem and Dana are REALLY swell
- (in case they dont buy that) I may already have
a black-belt in Karate..
Id rather learn from one bird how to singthan
teach ten thousand stars how not to dance--e.e.
cummings
54What are the lessons of KB Track?
- If TLPlan did better than SHOP in ICP, then how
are we supposed to interpret it? - That TLPlan is a superior planning technology
over SHOP? - That the naturally available domain knowledge in
the competition domains is easier to encode as
linear temporal logic statements on state
sequences than as procedures in the SHOP
language? - That Fahiem Bacchus and Jonas Kvarnstrom are way
better at coming up with domain knowledge for
blocks world (and other competition domains) than
Dana Nau?
We are NOT asking the right questions
55Questions worth asking in KB planner comparisons
(IMHO)
- How easy/natural (for humans) is the language in
which the planner accepts control knowledge? - How easy is it to validate the control
knowledge being input to the planner? - Is the naturally available knowledge about a
specific domain easily encoded in the language
accepted by the planner? - Does the planner allow any expertise
behaviorsolving the problems even without any
control knowledge, but improving performance with
added control knowledge? (or is the control
knowledge tightly intertwined with the domain
physics?).
56How/Why the competition is not asking the right
questions
- The role of the knowledge-engineer is played by
the same person(s) who wrote the planner. So, the
question of how natural the specific language is
for third-party knowledge engineers is largely
unaddressed. - No reasonable time limits are placed on coming up
with the control knowledge. So, we dont learn
much (or anything) about whether or not naturally
available knowledge about a domain is easily
representable in the language accepted by the
planner.
57Some suggestions for change
- Recruit third-party volunteers who will play the
role of knowledge engineers for the KB planners. - Ideally, we would like to have the same people
writing the control knowledge for a given domain
for all the competing approaches (so one
knowledge engineer per domain rather than one
knowledge engineer per planner). - (Alternative to above) Specify the control
knowledge that is available, so all planners
encode the same general knowledge. - One idea might be to ask the designers of the
domains (e.g. David Smith and his cohorts for the
Satellite and Rovers domain) to provide, in
english, what sort of control information they
would like the planner to use. - Measure the time taken to write and validate the
control knowledge. - Analyze the knowledge encoded by the different KB
planners for the same domain - Characterize it in terms of (a) whether the
knowledge is procedural or declarative and (b)
how hard would it be to learn the same
knowledge.
58Slides from here-on about learning domain
knowledge were not used
59Automated Customization of Planners
- Domain pre-processing
- Invariant detection Relevance detection
- Choice elimination, Type analysis
- STAN/TIM, DISCOPLAN etc.
- RIFO ONLP
- Abstraction
- ALPINE ABSTRIPS, STAN/TIM etc.
- Learning Search Control rules
- UCPOPEBL,
- PRODIGYEBL, (GraphplanEBL)
- Case-based planning (plan reuse)
- DerSNLP, Prodigy/Analogy
Most of the work has been done in the context of
Conjunctive Refinement Planners
Read Terry Zimmermans Learning
assisted planning Looking back, Taking Stock and
Going Forward
60Symmetry Invariant Detection
- Generate potential invariants and test them
- DISCOPLAN Gerevini et. al.
- Allows detection of higher-order mutexes
- Rintanens planner
- Uses model-verification techniques
- STAN/TIM
- Type analysis of the domain is used to generate
invariants - ONLP (Peot Smith)
- Use operator graph analysis to eliminate
non-viable choices
61Abstraction
- Idea
- Abstract some details of the problem or actions.
- Solve the abstracted version.
- Extend the solution to the detailed version
- Precondition Abstraction
- Work on satisfying important preconditions first
- Importance judged by
- Length of plans for subgoals ABSTRIPS, PABLO
- Inter-goal relations ALPINE
- Distribution-based HighPoint
- Strong abstractions (with downward refinement
property) are rare - Effectiveness is planner-dependent
- Clashes with other heuristics such as most
constrained first
62Example Abstracting Resources
- Most planners thrash by addressing planning and
scheduling considerations together - Eg. Blocks world, with multiple robot hands
- Idea Abstract resources away during planning
- Plan assuming infinite resources
- Do a post-planning resource allocation phase
- Re-plan if needed
(with Biplav Srivastava)
63Learning Search Control Rules with EBL
Explain leaf level failures Regress the
explanations to compute interior node failure
explanations Use failure explanations to set up
control rules Problems -- Most branches
end in depth-limits gtNo
analytical explanation gtUse preference
rules? -- THE Utility problem gtLearn
general rules gtKeep usage statistics
prune useless rules
UCPOPEBL
If Polished(x)_at_S Initially-True(Polished(x))
Then REJECT
Stepadd(Roll(x),Cylindrical(x)_at_s)
(Kambhampati, Katukam, Qu, 95)
64Example Rules (Learned)
UCPOP
Prodigy
If (and (current-node node)
(candidate-goal node
(shape obj Cyl))
(candidate-goal node
(surface-condition obj
polished))) Then Prefer (shape obj cyl) to
(surface-condition obj polished)
If Polished(x)_at_S Initially-True(Polished(x))
Then REJECT
Stepadd(Roll(x),Cylindrical(x)_at_s)
Pruning rule
Preference rule
65Case-based Planning Macrops, Reuse, Replay
- Structures being reused
- Opaque vs. Modifiable
- Solution vs. Solving process (derivation)
- Acquisition of structures to be reused
- Human given vs. Automatically acquired
- Mechanics of reuse
- Phased vs. simultaneous
- Costs
- Storage Retrieval costs Solution quality
66 Case-study DerSNLP
( Ihrig Kambhampati, JAIR 97)
- Modifiable derivational traces are reused
- Traces are automatically acquired during problem
solving - Analyze the interactions among the parts of a
plan, and store plans for non-interacting
subgoals separately - Reduces retrieval cost
- Use of EBL failure analysis to detect
interactions - All relevant trace fragments are retrieved and
replayed before the control is given to
from-scratch planner - Extension failures are traced to individual
replayed traces, and their storage indices are
modified appropriately - Improves retrieval accuracy
Old cases
67DerSNLP Results
Performance with increased Training
Solvability with increased traning
Problem size (goals)
5
Library Size
3
1
(JAIR, 97)
68Reuse in Disjunctive Planning
- Harder to make a disjunctive planner commit to
extending a specific plan first - Options
- Support opaque macros along with primitive
actions - Increases the size of k-step disjunctive plan
- But a solution may be found at smaller k
- Modify the problem/domain specification so the
old plans constraints will be respected in any
solution (Bailotti et. al.) - MAX-SAT formulations of reuse problem
- Constrain the encoding so that certain steps may
have smaller step-action mapping and ordering
choices - Causal encodings provide better support
with Amol Mali