Temporal Planning and Resource Allocation

1
Temporal Planning and Resource Allocation

• Stefanie Chiou, Rob Kochman, and Gary Look

2
Running Plans in the Real World

• Need to account for time and resources when
  creating plans
• Papers featured
• "Executing Reactive, Model-Based Programs through
  Graph-Based Temporal Planning" by Phil Kim, Brian
  C. Williams, and Mark Abramson (IJCAI 01)
• "Managing Multiple Tasks in Complex, Dynamic
  Environments" by Michael Freed (AAAI 98).

3
Paper

• Executing Reactive, Model-based Programs through
  Graph-based Temporal Planning by Phil Kim, Brian
  Williams, and Mark Abramson

4
Familiar Examples
Mars Climate Orbiter 12/11/98
Mars Polar Lander 1/3/99
5
Motivation

• Embedded programming is hard
• Easier to reason about state when programming

6
Overview/Contributions

• RMPL provides a new programming paradigm for
  programming robust systems of cooperative
  autonomous agents
• TPN -gt synthesis of temporal, causal link, and
  HTN planning
• A holy grail for autonomous agents
• Planner that implements these ideas

7
RMPL Intro

• RMPL supports four types of reasoning about
  system interactions
• reasoning about contingencies
• scheduling
• inferring hidden state
• controlling hidden state
• This paper focuses on first two interaction types

8
(Model-based) Embedded Programs

• Embedded programs interact withplant
  sensors/actuators
• Read sensors
• Set actuators

• Model-based programs interact with plant state
• Read state
• Write state

setState
getState
Programmer must map between state and
sensors/actuators.
Model-based executive maps between sensors,
actuators to states.
9
Model-based Embedded Program Breakdown
Model-basedEmbedded Program
setState
getState
Model-based executive maps between sensors,
actuators to states.
Model-based Executive
Sensor data
Actuator commands
S Plant
10
Example The model-based program sets engine
thrusting, and the deductive controller . . .
Fuel tank
Oxidizer tank
11
Time and Contingency Constructs in RMPL

• if c thennext A
• do A maintaining C
• A,B (concurrency)
• AB (serialization)
• Al,u (temporal bounds)
• ChooseA,B (choose)

12
RMPL Code Example

• Group-Enroute()l,u
• choose
• do
• Group-Fly-Path(PATH1) l90,u90
• maintaining PATH1_OK,
• do
• Group-Fly-Path(PATH2) l90,u90
• maintaining PATH2_OK
• Group-Transmit(FAC,ARRIVED_TAI)0,2,
• do
• Group-Wait(TAI_HOLD1,TAI_HOLD2)0,u10
• watching PROCEED_OK

13
RMPLs Representation of Time and Contingencies

• Important to find a plan quickly
• Idea use a plan graph
• Generalization of Simple Temporal Network (STN)
• TPN defined (STN conditionals choices)

14
STN example
Start
End



15
Temporal Planning Networks (TPN)

• A temporal planning network is just a
  generalization of a STN
• Includes ability to represent conditionals and
  choices

16
TPN Example
17
RMPL -gt TPN conversion

• A l,u invoke activity A between l and u time
  units

18
RMPL -gt TPN conversion

• c l,u Assert that condition c is true now
  until l ,u

19
RMPL -gt TPN conversion

• If c thennext A l,u Execute A for l ,u, if
  condition c is currently satisfied

20
RMPL -gt TPN conversion

• do A l,u maintaining c Execute A for
  l ,u, and ensure that condition c holds
  throughout

21
RMPL -gt TPN conversion

• A l1,u1, B l2,u2 Concurrently execute A for
  l1 ,u1, and B for l2 ,u2

22
RMPL -gt TPN conversion

• A l1,u1 B l2,u2 Execute A for l1 ,u1,
  and then B for l2 ,u2

23
RMPL -gt TPN conversion

• choose A l1,u1 B l2,u2 Reduces to
  A l1 ,u1 or B l2 ,u2 non-deterministically

24
Kirk

• Compiles RMPL program into a TPN
• Searches TPN for a temporally consistent plan
• Temporally consistent plan is embedded into the
  TPN.

25
Kirk Phase1

• Select plan from TPN
• Essentially a graph traversal
• Check plan for temporal consistency

Start
26
Selecting the Plan
Start
Start
27
Checking for Temporal Consistency

• Convert TPN to a distance graph
• Run Bellman-Ford to check for negative cycles (if
  any found, inconsistent)

28
Converting TPNs to Distance Graphs

• The interval aij,bij represents the statement
  aij Tj-Ti bij
• This is equivalent to Tj-Ti bij and Ti-Tj
  -aij

29
Checking for Temporal Consistency

• Convert TPN to a distance graph
• Run Bellman-Ford algorithm to check for negative
  cycles

30
Bellman-Ford Algorithm

• initializeCosts(G, s)
• for i1 to V(G)-1
• for each edge (u,v) in E(G)
• updateCost(u, v, w)
• for each edge (u, v) in E(G)
• if cost(v) gt cost(u) w(u. v)
• return false
• return true

31
Bellman-Ford Example
Source
32
Bellman-Ford Example
Source
33
Bellman-Ford Example
Source
34
Bellman-Ford Example
Source
35
Bellman-Ford Example
Source
36
Bellman-Ford Example
Source
37
Bellman-Ford Example
Source
38
Bellman-Ford Example
Source
39
Kirk Phase 2

• Resolve threats and open conditions
• Analogous to threats and open conditions in
  causal link planning
• Identify intervals of inconsistent constraints
  using Floyd-Warshall
• Order intervals to resolve threats
• Close open conditions by making sure open
  conditions satisfied by some action in the plan

40
Why This Paper?

• Its useful for our term project

41
Vision

• "Managing Multiple Tasks in Complex, Dynamic
  Environments" by Michael Freed (AAAI 98).
• Achieve goals in task environments
• Complex
• Time-pressured
• Uncertain
• Co-existing/Interacting

42
APEX Goal ATC

• Goal simulate human air traffic controllers
• Largely routine activity
• Complexity due to many simple tasks
• Interruptions necessary

43
APEX Goal ATC
44
APEX Goal ATC
45
APEX Goal ATC
46
Resource Conflicts

• Separate tasks make incompatible demands
• What to do?
• Determine relative priority of tasks
• Assign control to winner
• Deal with the loser

47
Conflict Resolution Strategies

• Shed
• Eliminate low importance tasks
• When (Demand gt Availability)
• Delay/Interrupt
• Introduces complications
• Circumvent
• Select methods that use different resources

48
APEX Architecture Two Parts

• Resource Architecture
• Set of resources
• Cognitive
• Perceptual
• Motor
• Action Selection Component

Action Selection Component
commands
events
Resource Architecture
perception
actuators
World
49
Procedure Definition Language (PDL)
Example Turning on headlights
50
Procedure Definition Language (PDL)
Example Turning on headlights

• (procedure
• (index (turn-on-headlights)
• (step s1 (clear-hand left-hand))
• (step s2 (determine-loc headlight-ctl gt
  ?loc))
• (step s3 (grasp knob left-hand ?loc)
• (waitfor ?s1 ?s2))
• (step s4 (pull knob left-hand 0.4) (waitfor
  ?s3))
• (step s5 (ungrasp left-hand) (waitfor ?s4))
• (step s6 (terminate) (waitfor ?s5)))

51
Detecting Conflicts

• Must determine
• Which tasks should be checked and when
• Preconditions satisfied
• Resources become available
• Whether conflict exists between specified tasks
• Direct and indirect control

52
PROFILE Clause

• Denotes resource requirements for a procedure

(profile (ltresourcegt ltdurationgt
ltcontinuitygt)) (profile (left-hand 8 10))
53
Prioritization of Tasks

• Used when
• New resource conflict detected
• New information potentially changes a previous
  prioritization decision

54
Prioritization Example Reprioritize

• (procedure
• (index (drive-car))
• . . .
• (step s8 (monitor-behind))
• (step s9 (reprioritize ?s8)
• (waitfor (sound-type ?sound car-horn)
• (loudness ?sound ?db (?if (gt ?db 30))))))
• (urgency ?y)))

55
Assigning Priority

• (step s5 (monitor-fuel-gauge) (priority 3))
• (step s6 (monitor-left-traffic) (priority ?x))
• (step s7 (monitor-ahead) (priority ( ?x ?y)))

56
General Priority Form

• (priority ltbasisgt (importance ltexpressiongt)
• (urgency ltexpressiongt))
• (step s5 (monitor-fuel-gauge)
• (priority (run-empty) (importance 6) (urgency
  2))
• (priority (delay-to-other-task) (importance ?x)
• (urgency 3))
• (priority (excess-time-cost refuel) (importance
  ?x)
• (urgency ?y)))

57
Importance vs. Urgency

• Depends on workload
• priorityb SIb (Smax-S)Ub
• S is subjective workload (a heuristic
  approximation of actual workload)
• Ib and Ub represent importance and urgency for a
  specified basis

58
Interruption RESET

• . . .
• (step s4 (turn-on-headlights))
• (step s5 (reset) (waitfor (suspended ?s4))

59
Coping with Interruption

• Wind-down activities
• Suspension-time activities
• Wind-up activities

60
Wind-down Activities an Example

• (step s15 (pull-over)
• (waitfor (suspended ?self))
• (priority (avoid-accident) (importance 10)
• (urgency 10)))

61
Interrupt Costs

• Wind-down, suspension, and wind-up activities
  incur cost
• Ongoing task has its priority increased in
  proportion to interrupt cost
• (interrupt-cost 5)

62
Slack Time

• (step s17 (suspend ?self)
• (waitfor (shape ?object traffic-signal)
• (color ?object red)))
• (step s18 (monitor-object ?object) (waitfor
  ?s17))
• (step s19 (reprioritize ?self)
• (waitfor (color ?object green)))

63
Computing Priority
IC interruption cost U urgency I
importance S workload
64
Conflict Resolution Strategies

• Shed
• Eliminate low importance tasks
• When (Demand gt Availability)
• Delay/Interrupt
• Introduces complications
• Circumvent
• Select methods that use different resources

65
Evaluation and Future Work

• Strengths and Weaknesses
• ATC application has identified issues
• Computing overall priority from base priorities
• Suppression of base priorities
• Other priority issues
• A, B need X
• A, C need Y
• Priority of A must exceed that of BC