Chapter 17 Planning Based on Model Checking - PowerPoint PPT Presentation

1 / 42
About This Presentation
Title:

Chapter 17 Planning Based on Model Checking

Description:

Automated Planning: Theory and Practice Chapter 17 Planning Based on Model Checking Dana S. Nau University of Maryland * * ' {(s,a) : a is applicable to s ... – PowerPoint PPT presentation

Number of Views:136
Avg rating:3.0/5.0
Slides: 43
Provided by: Dana124
Learn more at: http://www.cs.umd.edu
Category:

less

Transcript and Presenter's Notes

Title: Chapter 17 Planning Based on Model Checking


1
Chapter 17Planning Based on Model Checking
Lecture slides for Automated Planning Theory and
Practice
  • Dana S. Nau
  • University of Maryland
  • 1123 PM June 16, 2015

2
Motivation
c
a
b
Intendedoutcome
  • Actions with multiple possibleoutcomes
  • Action failures
  • e.g., gripper drops its load
  • Exogenous events
  • e.g., road closed
  • Nondeterministic systems are like Markov Decision
    Processes (MDPs), but without probabilities
    attached to the outcomes
  • Useful if accurate probabilities arent
    available, or if probability calculations would
    introduce inaccuracies

c
grasp(c)
a
b
c
a
b
Unintendedoutcome
3
Nondeterministic Systems
  • Nondeterministic system a triple ? (S, A, ?)
  • S finite set of states
  • A finite set of actions
  • ? S ? A ? 2s
  • Like in the previous chapter, the book doesnt
    commit to any particular representation
  • It only deals with the underlying semantics
  • Draw the state-transition graph explicitly
  • Like in the previous chapter, a policy is a
    function from states into actions
  • p S ? A
  • Notation Sp s (s,a) ? p
  • In some algorithms, well temporarily have
    nondeterministic policies
  • Ambiguous multiple actions for some states
  • p S ? 2A, or equivalently, p ? S ? A
  • Well always make these policies deterministic
    before the algorithm terminates

4
Example
  • Robot r1 startsat location l1
  • Objective is toget r1 to location l4
  • p1 (s1, move(r1,l1,l2)), (s2,
    move(r1,l2,l3)), (s3, move(r1,l3,l4))
  • p2 (s1, move(r1,l1,l2)), (s2,
    move(r1,l2,l3)), (s3, move(r1,l3,l4)), (s5,
    move(r1,l3,l4))
  • p3 (s1, move(r1,l1,l4))

5
Example
  • Robot r1 startsat location l1
  • Objective is toget r1 to location l4
  • p1 (s1, move(r1,l1,l2)), (s2,
    move(r1,l2,l3)), (s3, move(r1,l3,l4))
  • p2 (s1, move(r1,l1,l2)), (s2,
    move(r1,l2,l3)), (s3, move(r1,l3,l4)), (s5,
    move(r1,l3,l4))
  • p3 (s1, move(r1,l1,l4))

6
Example
  • Robot r1 startsat location l1
  • Objective is toget r1 to location l4
  • p1 (s1, move(r1,l1,l2)), (s2,
    move(r1,l2,l3)), (s3, move(r1,l3,l4))
  • p2 (s1, move(r1,l1,l2)), (s2,
    move(r1,l2,l3)), (s3, move(r1,l3,l4)), (s5,
    move(r1,l3,l4))
  • p3 (s1, move(r1,l1,l4))

7
Execution Structures
s5
s2
s3
  • Execution structurefor a policy p
  • The graph of all ofps execution paths
  • Notation ?p (Q,T)
  • Q ? S
  • T ? S ? S
  • p1 (s1, move(r1,l1,l2)), (s2,
    move(r1,l2,l3)), (s3, move(r1,l3,l4))
  • p2 (s1, move(r1,l1,l2)), (s2,
    move(r1,l2,l3)), (s3, move(r1,l3,l4)), (s5,
    move(r1,l3,l4))
  • p3 (s1, move(r1,l1,l4))

s1
s4
8
Execution Structures
s5
s2
s3
  • Execution structurefor a policy p
  • The graph of all ofps execution paths
  • Notation ?p (Q,T)
  • Q ? S
  • T ? S ? S
  • p1 (s1, move(r1,l1,l2)), (s2,
    move(r1,l2,l3)), (s3, move(r1,l3,l4))
  • p2 (s1, move(r1,l1,l2)), (s2,
    move(r1,l2,l3)), (s3, move(r1,l3,l4)), (s5,
    move(r1,l3,l4))
  • p3 (s1, move(r1,l1,l4))

s1
s4
9
Execution Structures
s5
s2
s3
  • Execution structurefor a policy p
  • The graph of all ofps execution paths
  • Notation ?p (Q,T)
  • Q ? S
  • T ? S ? S
  • p1 (s1, move(r1,l1,l2)), (s2,
    move(r1,l2,l3)), (s3, move(r1,l3,l4))
  • p2 (s1, move(r1,l1,l2)), (s2,
    move(r1,l2,l3)), (s3, move(r1,l3,l4)), (s5,
    move(r1,l3,l4))
  • p3 (s1, move(r1,l1,l4))

s1
s4
10
Execution Structures
s5
s2
s3
  • Execution structurefor a policy p
  • The graph of all ofps execution paths
  • Notation ?p (Q,T)
  • Q ? S
  • T ? S ? S
  • p1 (s1, move(r1,l1,l2)), (s2,
    move(r1,l2,l3)), (s3, move(r1,l3,l4))
  • p2 (s1, move(r1,l1,l2)), (s2,
    move(r1,l2,l3)), (s3, move(r1,l3,l4)), (s5,
    move(r1,l3,l4))
  • p3 (s1, move(r1,l1,l4))

s1
s4
11
Execution Structures
  • Execution structurefor a policy p
  • The graph of all ofps execution paths
  • Notation ?p (Q,T)
  • Q ? S
  • T ? S ? S
  • p1 (s1, move(r1,l1,l2)), (s2,
    move(r1,l2,l3)), (s3, move(r1,l3,l4))
  • p2 (s1, move(r1,l1,l2)), (s2,
    move(r1,l2,l3)), (s3, move(r1,l3,l4)), (s5,
    move(r1,l3,l4))
  • p3 (s1, move(r1,l1,l4))

s1
s4
12
Execution Structures
  • Execution structurefor a policy p
  • The graph of all ofps execution paths
  • Notation ?p (Q,T)
  • Q ? S
  • T ? S ? S
  • p1 (s1, move(r1,l1,l2)), (s2,
    move(r1,l2,l3)), (s3, move(r1,l3,l4))
  • p2 (s1, move(r1,l1,l2)), (s2,
    move(r1,l2,l3)), (s3, move(r1,l3,l4)), (s5,
    move(r1,l3,l4))
  • p3 (s1, move(r1,l1,l4))

s1
s4
13
Types of Solutions
  • Weak solution at least one execution path
    reaches a goal
  • Strong solution every execution path reaches a
    goal
  • Strong-cyclic solution every fair execution path
    reaches a goal
  • Dont stay in a cycle forever if theres a
    state-transition out of it

s0
s3
Goal
a2
a0
s2
s1
Goal
a3
a1
a3
s0
s3
a2
a0
Goal
s2
s1
a1
14
Finding Strong Solutions
  • Backward breadth-first search
  • StrongPreImg(S) (s,a) ?(s,a) ? ?,
    ?(s,a) ? S
  • all state-action pairs for whichall of the
    successors are in S
  • PruneStates(p,S) (s,a) ? p s ? S
  • S is the set of states wevealready solved
  • keep only the state-actionpairs for other states
  • MkDet(p')
  • p' is a policy that may be nondeterministic
  • remove some state-action pairs ifnecessary, to
    get a deterministic policy

15
Example
2
  • p failure
  • p' ?
  • Sp' ?
  • Sg ? Sp' s4

Start
s4
Goal
16
Example
s5
2
  • p failure
  • p' ?
  • Sp' ?
  • Sg ? Sp' s4
  • p'' ? PreImage (s3,move(r1,l3,l4)),
    (s5,move(r1,l5,l4))

s3
Start
s4
Goal
17
Example
s5
2
  • p failure
  • p' ?
  • Sp' ?
  • Sg ? Sp' s4
  • p'' ? PreImage (s3,move(r1,l3,l4)),
    (s5,move(r1,l5,l4))
  • p ? p' ?
  • p' ? p' U p'' (s3,move(r1,l3,l4)),
    (s5,move(r1,l5,l4))

s3
Start
s4
Goal
18
Example
s5
  • p ?
  • p' (s3,move(r1,l3,l4)),
    (s5,move(r1,l5,l4))
  • Sp' s3,s5
  • Sg ? Sp' s3,s4,s5

2
s3
Start
s4
Goal
19
Example
s5
  • p ?
  • p' (s3,move(r1,l3,l4)),
    (s5,move(r1,l5,l4))
  • Sp' s3,s5
  • Sg ? Sp' s3,s4,s5
  • PreImage ? (s2,move(r1,l2,l3)),
    (s3,move(r1,l3,l4)), (s5,move(r1,l5,l4)),
    (s3,move(r1,l4,l3)), (s5,move(r1,l4,l5))
  • p'' ? (s2,move(r1,l2,l3))
  • p ? p' (s3,move(r1,l3,l4)),
    (s5,move(r1,l5,l4))
  • p' ? (s2,move(r1,l2,l3),
    (s3,move(r1,l3,l4)), (s5,move(r1,l5,l4))

2
s2
s3
Start
s4
Goal
20
Example
s5
  • p (s3,move(r1,l3,l4)), (s5,move(r1,l5,l4))
  • p' (s2,move(r1,l2,l3)),
    (s3,move(r1,l3,l4)), (s5,move(r1,l5,l4))
  • Sp' s2,s3,s5
  • Sg ? Sp' s2,s3,s4,s5

2
s2
s3
Start
s4
Goal
21
Example
s5
  • p (s3,move(r1,l3,l4)), (s5,move(r1,l5,l4))
  • p' (s2,move(r1,l2,l3)),
    (s3,move(r1,l3,l4)), (s5,move(r1,l5,l4))
  • Sp' s2,s3,s5
  • Sg ? Sp' s2,s3,s4,s5
  • p'' ? (s1,move(r1,l1,l2))
  • p ? p' (s2,move(r1,l2,l3)),
    (s3,move(r1,l3,l4)), (s5,move(r1,l5,l4))
  • p' ? (s1,move(r1,l1,l2)),
    (s2,move(r1,l2,l3)),
    (s3,move(r1,l3,l4)),
    (s5,move(r1,l5,l4))

2
s2
s3
Start
s1
s4
Goal
22
Example
s5
2
  • p (s2,move(r1,l2,l3)), (s3,move(r1,l3,l4)),
    (s5,move(r1,l5,l4))
  • p' (s1,move(r1,l1,l2)),
    (s2,move(r1,l2,l3)), (s3,move(r1,l3,l4)),
    (s5,move(r1,l5,l4))
  • Sp' s1,s2,s3,s5
  • Sg ? Sp' s1,s2,s3,s4,s5

s2
s3
Start
s1
s4
Goal
23
Example
s5
2
  • p (s2,move(r1,l2,l3)), (s3,move(r1,l3,l4)),
    (s5,move(r1,l5,l4))
  • p' (s1,move(r1,l1,l2)),
    (s2,move(r1,l2,l3)), (s3,move(r1,l3,l4)),
    (s5,move(r1,l5,l4))
  • Sp' s1,s2,s3,s5
  • Sg ? Sp' s1,s2,s3,s4,s5
  • S0 ? Sg ? Sp'
  • MkDet(p') p'

s2
s3
Start
s1
s4
Goal
24
Finding Weak Solutions
  • Weak-Plan is just like Strong-Plan except for
    this
  • WeakPreImg(S) (s,a) ?(s,a) i S ? ?
  • at least one successor is in S

Weak
Weak
25
Example
2
  • p failure
  • p' ?
  • Sp' ?
  • Sg ? Sp' s4

Start
s4
Goal
Weak
Weak
26
Example
s5
2
  • p failure
  • p' ?
  • Sp' ?
  • Sg ? Sp' s4
  • p'' PreImage (s1,move(r1,l1,l4)),
    (s3,move(r1,l3,l4)), (s5,move(r1,l5,l4))
  • p ? p' ?
  • p' ? p' U p'' (s1,move(r1,l1,l4)),
    (s3,move(r1,l3,l4)), (s5,move(r1,l5,l4))

s3
Start
s1
s4
Goal
Weak
Weak
27
Example
s5
2
  • p ?
  • p' (s1,move(r1,l1,l4)),
    (s3,move(r1,l3,l4)), (s5,move(r1,l5,l4))
  • Sp' s1,s3,s5
  • Sg ? Sp' s1,s3,s4,s5
  • S0 ? Sg ? Sp'
  • MkDet(p') p'

s3
Start
s1
s4
Goal
Weak
Weak
28
Finding Strong-Cyclic Solutions
  • Begin with a universal policy p' that contains
    all state-action pairs
  • Repeatedly, eliminate state-action pairs that
    take us to bad states
  • PruneOutgoing removes state-action pairs that go
    to states not in Sg?Sp
  • PruneOutgoing(p,S) p (s,a) ? p ?(s,a) ?
    S?Sp
  • PruneUnconnected removes states from which it is
    impossible to get to Sg
  • Start with p' ?, compute fixpoint of p' ? p n
    WeakPreImg(Sg?Sp)

29
FindingStrong-Cyclic Solutions
s5
2
s2
s3
  • Once the policy stops changing,
  • If its not a solution, returnfailure
  • RemoveNonProgressremoves state-actionpairs
    that dont gotoward the goal
  • implement asbackward searchfrom the goal
  • MkDet makes suretheres only one actionfor each
    state

Start
s1
s4
s6
Goal
at(r1,l6)
30
Example 1
s5
2
  • p ? ?
  • p' ? (s,a) a is applicable to s

s2
s3
Start
s1
s4
s6
Goal
at(r1,l6)
31
Example 1
s5
2
  • p ? ?
  • p' ? (s,a) a is applicable to s
  • p ? (s,a) a is applicable to s
  • PruneOutgoing(p',Sg) p'
  • PruneUnconnected(p',Sg) p'
  • RemoveNonProgress(p') ?

s2
s3
Start
s1
s4
s6
Goal
at(r1,l6)
32
Example 1
s5
2
  • p ? ?
  • p' ? (s,a) a is applicable to s
  • p ? (s,a) a is applicable to s
  • PruneOutgoing(p',Sg) p'
  • PruneUnconnected(p',Sg) p'
  • RemoveNonProgress(p') as shown

s2
s3
Start
s1
s4
s6
Goal
at(r1,l6)
33
Example 1
s5
2
  • p ? ?
  • p' ? (s,a) a is applicable to s
  • p ? (s,a) a is applicable to s
  • PruneOutgoing(p',Sg) p'
  • PruneUnconnected(p',Sg) p'
  • RemoveNonProgress(p') as shown
  • MkDet() either(s1,move(r1,l1,l4),
    (s2,move(r1,l2,l3)), (s3,move(r1,l3,l4),
    (s4,move(r1,l4,l6), (s5,move(r1,l5,l4)
  • or (s1,move(r1,l1,l2), (s2,move(r1,l2,l3)),
    (s3,move(r1,l3,l4), (s4,move(r1,l4,l6),
    (s5,move(r1,l5,l4)

s2
s3
Start
s1
s4
s6
Goal
at(r1,l6)
34
Example 2 no applicable actions at s5
s5
2
  • p ? ?
  • p' ? (s,a) a is applicable to s

s2
s3
Start
s1
s4
s6
Goal
at(r1,l6)
35
Example 2 no applicable actions at s5
s5
2
  • p ? ?
  • p' ? (s,a) a is applicable to s
  • p ? (s,a) a is applicable to s
  • PruneOutgoing(p',Sg)

s2
s3
Start
s1
s4
s6
Goal
at(r1,l6)
36
Example 2 no applicable actions at s5
s5
2
  • p ? ?
  • p' ? (s,a) a is applicable to s
  • p ? (s,a) a is applicable to s
  • PruneOutgoing(p',Sg) p'

s2
s3
Start
s1
s4
s6
Goal
at(r1,l6)
37
Example 2 no applicable actions at s5
2
  • p ? ?
  • p' ? (s,a) a is applicable to s
  • p ? (s,a) a is applicable to s
  • PruneOutgoing(p',Sg) p'
  • PruneUnconnected(p',Sg) as shown

s2
s3
Start
s1
s4
s6
Goal
at(r1,l6)
38
Example 2 no applicable actions at s5
2
  • p ? ?
  • p' ? (s,a) a is applicable to s
  • p ? (s,a) a is applicable to s
  • PruneOutgoing(p',Sg) p'
  • PruneUnconnected(p',Sg) as shown
  • p' ? as shown

s2
s3
Start
s1
s4
s6
Goal
at(r1,l6)
39
Example 2 no applicable actions at s5
2
  • p' ? as hown
  • p ? p'
  • PruneOutgoing(p',Sg) p'
  • PruneUnconnected(p',Sg) p'
  • so p p'
  • RemoveNonProgress(p')

s2
s3
Start
s1
s4
s6
Goal
at(r1,l6)
40
Example 2 no applicable actions at s5
2
  • p' ? shown
  • p ? p'
  • PruneOutgoing(p',Sg) p'
  • PruneUnconnected(p'',Sg) p'
  • so p p'
  • RemoveNonProgress(p') as shown
  • MkDet(shown) no change

s2
s3
Start
s1
s4
s6
Goal
at(r1,l6)
41
Planning for Extended Goals
  • Here, extended means temporally extended
  • Constraints that apply to some sequence of states
  • Examples
  • want to move to l3,and then to l5
  • want to keep goingback and forthbetween l3 and
    l5

42
Planning for Extended Goals
  • Context the internal state of the controller
  • Plan (C, c0, act, ctxt)
  • C a set of execution contexts
  • c0 is the initial context
  • act S ? C ? A
  • ctxt S ? C ? S ? C
  • Sections 17.3 extends the ideas in Sections 17.1
    and 17.2 to deal with extended goals
  • Well skip the details
Write a Comment
User Comments (0)
About PowerShow.com