Knowledge Repn'

1
Knowledge Repn. ReasoningLec 26 Filtering
with Logic

• UIUC CS 498 Section EA
• Professor Eyal AmirFall Semester 2004

2
Last Time

• Dynamic Bayes Nets
• Forward-backward algorithm
• Filtering
• Approximate inference via factoring and sampling

3
Filtering Stochastic Processes

• Dynamic Bayes Nets (DBNs) factored representation

4
Filtering Stochastic Processes

• Dynamic Bayes Nets (DBNs) factored representation

s4
s3
s2
s5
s4
s3
s2
s5
s4
s3
s2
s5
s4
s3
s2
s5
5
Filtering Stochastic Processes

• Dynamic Bayes Nets (DBNs) factored representation

s4
s3
s5
s4
s3
s5
s4
s3
s5
s4
s3
s5
6
Filtering Stochastic Processes

• Dynamic Bayes Nets (DBNs) factored representation

s4
s5
O(2n) space O(22n) time
s4
s5
s4
s5
s4
s5
7
Filtering Stochastic Processes

• Dynamic Bayes Nets (DBNs) factored
  representation O(2n) space, O(22n) time
• Kalman Filter Gaussian belief state and linear
  transition model

8
Filtering Stochastic Processes

• Dynamic Bayes Nets (DBNs) factored
  representation O(2n) space, O(22n) time
• Kalman Filter Gaussian belief state and linear
  transition model

s4
s5
O(n2) space O(n3) time
s4
s5
s4
s5
9
Complexity Results

• Filtering for deterministic systems is NP-hard
  when the initial state is not fully known
  Liberatore 97

• AmirRussell03 Every representation of
• belief states grows exponentially for
• some deterministic systems

10
Today

• Tracking and filtering logical knowledge
• Foundations for efficient filtering
• Compact representation indefinitely
• Possible projects

11
Logical Filtering

• Belief state logical formula

12
Logical Filtering

• Belief state logical formula
• Observations logical formulae

13
Logical Filtering

• Belief state logical formula
• Observations logical formulae
• Actions effect rules
• e.g., fetch(X,Y) causes has(X) if in(X,Y)

14
Logical Filtering

• Belief state logical formula
• Observations logical formulae
• Actions effect rules
• e.g., fetch(X,Y) causes has(X) if in(X,Y)
• Actions may be nondeterministic
• Partial observations

15
Example A Cleaning Robot

• Initial Knowledge
• ?
• Apply action
• fetch(broom,closet)
• Resulting knowledge
• Ø in(broom,closet)
• Reason
• If initially Øin(broom,closet), then still
  Øin(broom,closet)
• If initially in(broom,closet), then now
  Øin(broom,closet)

16
Filtering with Possible Worlds
Problem n world features ? 2n states
17
Filtering Possible Worlds

• Initially we are in s1,,sk
• Action a
• Filtera(s1,,sk)
• s R(s1,a,s) or or R(sk,a,s)
• observing o
• Filtero(s1,,su)
• s1,,su ? s o holds in s

18
Filtering with Logical Formulae

• Action-Definition(a)t,t1 ?
• Ù (Precondi(a)t ? Effecti(a)t1)
• i Ù Frame-Axioms(a)

19
Filtering with Logical Formulae

• Belief state S represented by j
• Actions Filtera(jt) ? logical resultst1 of
• jt Ù Action-Definition(a)t,t1

20
Filtering with Logical Formulae

• Belief state S represented by j
• Actions Filtera(jt) ? logical resultst1 of
• jt Ù Action-Definition(a)t,t1
• Observations Filtero(j) j Ù o
• jt1 Filtero(Filtera(jt))

21
Filtering with Logical Formulae

• Belief state S represented by j
• Actions Filtera(jt) ? logical resultst1 of
• jt Ù Action-Definition(a)t,t1
• Observations Filtero(j) j Ù o
• jt1 Filtero(Filtera(jt))
• Theorem formula filtering implements
  possible-worlds semantics

22
Contents

• Tracking and filtering logical knowledge
• Foundations for efficient filtering
• Compact representation indefinitely
• Possible projects

23
Distribution Properties

• Filtera(jÚy) º Filtera(j) Ú Filtera(y)
• Filtering a DNF belief state by factoring

24
Distribution Properties

• Filtera(jÚy) º Filtera(j) Ú Filtera(y)
• Filtera(jÙy) º Filtera(j) Ù Filtera(y)
• Filtera(Øj) º
• ØFiltera(j) Ù Filtera(TRUE)
• Filtering a DNF belief state by factoring

25
Distribution for Some Actions

• Filtera(jÚy) º Filtera(j) Ú Filtera(y)
• Filtera(jÙy) º Filtera(j) Ù Filtera(y)
• Filtera(Øj) º
• ØFiltera(j) Ù Filtera(TRUE)
• Filter literals in the belief-state formula
  separately, and combine the results

• STRIPS Actions
• 11 Actions

26
Actions that map states 11

• Examples
• flip(light) but not turn-on(light)
• increase(speed,10) but not set(speed,50)
• pickUp(X,Y) but not pickUp(X)
• Most actions are 11 in proper formulation

27
Actions that map states 11

• Reason for distribution over Ù

Filtera(jÙy) º Filtera(j) Ù Filtera(y)
11
Non-11
28
STRIPS Actions

• Possibly nondeterministic effects
• No conditions on effects
• Example turn-on(light)
• Used extensively in planning

29
Distribution for Some Actions

• Filtera(jÚy) º Filtera(j) Ú Filtera(y)
• Filtera(jÙy) º Filtera(j) Ù Filtera(y)
• Filtera(Øj) º
• ØFiltera(j) Ù Filtera(TRUE)
• Filter literals in the belief-state formula
  separately, and combine the results

• STRIPS Actions
• 11 Actions

30
Example Filtering a Literal

• Initial knowledge
• in(broom,closet)
• Apply fetch(broom,closet)
• Preconds in(broom,closet) Ù Ølocked(closet)
• Effects has(broom) Ù Øin(broom,closet)
• Resulting knowledge
• has(broom) Ù Øin(broom,closet) Ú
• locked(closet)

31
Example Filtering a Formula

• Initial knowledge
• in(broom,closet) Ù Ølocked(closet)
• Apply fetch(broom,closet)
• Preconds in(broom,closet) Ù Ølocked(closet)
• Effects has(broom) Ù Øin(broom,closet)
• Resulting knowledge
• has(broom) Ù Øin(broom,closet) Ù
  Ølocked(closet)

32
Filtering a Single Literal

• Closed-form solution
• Filtera(literal) Ù (Eff1 Ú ... Ú Effu) Ù B(a)
• literal Pre1 Ú ... Ú Preu
• a has effect rules (and frame rules) a causes
  Effi if Prei
• Eff1 ... Effu - effects of action a
• Pre1 ... Preu - preconditions of action a
• Roughly, B(a) ? Filtera(TRUE)

33
Filtering a Literal

• Filtera(literal) Ù (Eff1 Ú ... Ú Effu) Ù
  B(a)
• literal Pre1 Ú ... Ú Preu
• Belief state (j) Ø locked(closet)
• Action (a) fetch(broom,closet) with
• fetch(X,Y) causes has(X) Ù Ø in(X,Y) if
  Ø locked(Y) Ù in(X,Y)
• Belief state after a
• Filtera(Ølocked(closet))
• Ø locked(closet) Ù Ø in(broom,closet)

34
Filtering a Literal

• Filtera(literal) Ù (Eff1 Ú ... Ú Effu) Ù
  B(a)
• literal Pre1 Ú ... Ú Preu
• Action (a) fetch(broom,closet) with
• fetch(X,Y) causes has(X) Ù Ø in(X,Y) if
  Ø locked(Y) Ù in(X,Y)
• Belief state after a Filtera(Ølocked(closet))
  
• Ø locked(closet) Ù Ø in(broom,closet)
• Reason Ø locked(closet)
• (Ø locked(closet) Ù in(broom,closet)) Ú
• Ø in(broom,closet)

35
Algorithm for Permutation Actions

• Belief state (j)
• Ølocked(closet) Ù (in(broom,closet) Ú
  in(broom,shed))
• Action (a) fetch(broom,closet) with fetch(X,Y)
  causes
• has(X) Ù Øin(X,Y) if Ølocked(Y) Ù in(X,Y)
• Resulting belief state
• Filtera(j) Filtera(Ølocked(closet)) Ù
• (Filtera(in(broom,closet)) Ú
  Filtera(in(broom,shed)))
• Filtera(Ølocked(closet))
• Ølocked(closet) Ù Øin(broom,closet)

36
Algorithm for Permutation Actions

• Belief state (j)
• Ølocked(closet) Ù (in(broom,closet) Ú
  in(broom,shed))
• Filtera(Ølocked(closet))
• Ølocked(closet) Ù Øin(broom,closet)
• Filtera(in(broom,closet)) (Ølocked(closet) Ù
  Øin(broom,closet) Ù has(broom)) Ú
  in(broom,closet)
• Filtera(in(broom,shed)) in(broom,shed)
• Filtera(j) Ølocked(closet) Ù
  Øin(broom,closet) Ù (has(broom) Ú in(broom,shed))

37
Summary Efficient Update

• Fast exact update with any observation formulae,
  if one of the following
• STRIPS action (possibly nondeterministic)
• Action is a 11 mapping between states
• Belief states include all their prime implicates

38
Talk Outline

• Tracking and filtering knowledge
• Tractability results
• Compact representation over time
• Discussion Future work

39
Tractability and Representation Size

• Theorem1 Every propositional repn. of the belief
  state grows exponentially for some systems, even
  when initial belief state is compactly
  represented (follows from Boppana Sipser 90)

1 Rough statement. Complete one in A. Russell
03.
40
Example A Cleaning Robot

• Initial Knowledge
• in(broom,closet) Ú in(broom,shed)
• Apply action fetch(broom,closet)
• Resulting knowledge
• (has(broom) Ù Ølocked(closet) Ù
  Øin(broom,closet)) Ú
• (Øhas(broom) Ù locked(closet) Ù in(broom,closet))
  Ú
• (Øhas(broom) Ù in(broom,shed))
• Reason for space explosion uncertainty of
  actions success and preconditions applied

41
Compact Tractable Cases

• Compact belief state representation
• STRIPS actions with belief state in k-CNF
• 11 actions with belief state in k-CNF
• Observations in 2-CNF
• Theorem Filtering j with STRIPS actions
• k-CNF ? k-CNF
• time O(j 2rules(a))
• Corollary Filtering with STRIPS actions keeps
  belief state in O(nk) size (k fixed).

42
STRIPS-Filter Experiments
Average time per step
270 features
240 features
210 features
Filter time (m.sec)
180 features
150 features
Filtering step
43
STRIPS-Filter Experiments
Average space per step
210 features
185 features
160 features
135 features
Filter space (literals)
110 features
Filtering step
44
Intuition for More Results

• Filtering with deterministic action a is
  equivalent to filtering with actions a1 (a
  succeeds) or a2 (a fails) successfully,
• a1,a2 STRIPS with known success/failure
• Filtera(f) ? Filtera1(f) v Filtera2(f)
• STRIPS with known success/failure
• Filtera(l1Ú...Úlu) (l1Ú...Úlu) Ù B(a) or
• B(a)

45
Recent Results (unpublished) 1

• Compact representation indefinitely for STRIPS,
  if failure leaves features unchanged, and effects
  are 2-clauses
• a causes  (f v g) (g v -h)  if  x y
• Starting from belief state with r clauses we get
  at most max(r,n) clauses indefinitely, if effects
  are conjunction of at most two clauses

46
Recent Results (unpublished) 2

• Compact representation indefinitely for STRIPS,
  if failure has nondeterministic effect on
  affected features
• a causes  f g  if  x y
• a causes (f v -f) (g v -g) if (-x v -y)
• Belief state in k-CNF maintained indefinitely, if
  effects in k1-CNF, preconditions in k2-DNF,
  kk1k2

47
Related Work

• Stochastic filtering
• Kalman 60, Doucet et-al. 00, Dean
  Kanazawa 88, Boyen Koller 98,
• Action theories and semantics
• Gelfond Lifschitz 97, Baral Son 01,
  Doherty et-al. 98,
• Computation of progression
• Winslett 90, del Val 92, Lin Reiter
  97, Simon del Val 01,

48
Possible Projects

• More families of actions/observations
• Stochastic conditions on observations
• Different data structures (BDDs? Horn?)
• Compact and efficient stochastic filtering
• Relational / first-order filtering
• Dynamic observation models, filtering in
  expanding worlds
• Logical Filtering of numerical variables

49
More Projects

• Filtering for Kriegspiel (partially observable
  chess)
• Autonomous exploration of uncharted domains
• Smart agents in rich environments

50
THE END
51
Example Explosion of Space

• Initial Knowledge
• in(broom,closet) Ú in(broom,shed)
• Apply action fetch(broom,closet)
• Resulting knowledge
• (has(broom) Ù Ølocked(closet)
• Ù Øin(broom,closet)) Ú
• (Øhas(broom) Ù locked(closet)
• Ù in(broom,closet)) Ú
• (Øhas(broom) Ù in(broom,shed))

52
Tractability Problem

• Formula filtering is NP-hard in general
• Actions Filtera(jt) ?
• Cn(jt Ù (Precond(a)t ? Effect(a)t1)
• Ù Frame-Axioms(a))
• Cn() Logical consequences of
• Specific cases?
• Approximation?

53
Example A Cleaning Robot

• Initial Knowledge
• ?
• Apply action
• fetch(broom,closet)
• Resulting knowledge
• Ø in(broom,closet)
• Reason
• If initially Øin(broom,closet), then still
  Øin(broom,closet)
• If initially in(broom,closet), then now
  Øin(broom,closet)

54
Filtering Beliefs

• Filtering Update knowledge of the world after
  actions and observations
• Stochastic filtering examples
• Dynamic Bayes Nets (DBNs) factored
  representation
• Kalman Filter Gaussian belief state and linear
  transition model

55
Agents Acting in The World

• Agents in partially observable domains
• Cognitive, medical assistants
• Cleaning, gardening robots
• Space robots (exploration, repair, assist)
• Game-playing/companion agents
• Knowledgeable agents
• Use knowledge to decide on actions
• Update knowledge about the world

56
Example A Cleaning Robot

• Decides to clean the current room
• Knows the broom is in the closet
• Fetches the broom from the closet
• Now knows that the broom is in its hand and not
  in the closet

57
Filtering a Single Literal

• Closed-form solution
• Filtera(literal) Ù (Eff1 Ú ... Ú Effu) Ù B(a)
• literal Pre1 Ú ... Ú Preu
• a has effect rules (and frame rules) a causes
  Effi if Prei
• Eff1 ... Effu - effects of action a
• Pre1 ... Preu - preconditions of action a
• Roughly, B(a) ? Filtera(TRUE)

58
Permutation Actions

• Actions that permute the states
• flip(light) but not turn-on(light)
• increase(speed,10) but not set(speed,50)
• pickUp(X,Y) but not pickUp(X)

59
Results Tractable Cases

• Filtering a single literal
• Permutation actions
• STRIPS actions
• Prime-implicate representation of belief state

60
Filtering Logical FormulaeSTRIPS-Filter

• If every executed action was possible to execute
  (or we observed an error), and actions do not
  have conditional effects (but may have
  nondeterministic effects), and the belief state
  representation in PI-CNF, then
• Filtera( Ù Ci ) Ù Filtera(Ci)

61
Summary Tractable Cases

• Fast approximate update propositional belief
  state represented in NNF,CNF,DNF
• Fast exact update (if one of the following)
• Action is a 11 mapping between states
• STRIPS action (unconditional, nondeterministic
  effects of actions observations distinguish
  success from failure of action)
• Belief states include all their prime implicates
• Any observations

62
Sources of Difficulty for Compact Representation

• For action a with effect rule
• a causes Eff if Pre
• We always know after the action that
• Eff Ú ØPre
• If we know (Pre Ú p), then after the action we
  know
• Eff Ú p

63
How Is the State Kept Compact?

• STRIPS (nondeterministic) actions
• We always know that the precondition held (or we
  got a signal that the action failed)
• There are no conditional effects
• Permutation actions
• We restrict the preconditions and effects, e.g.,
• All rules of the form a causes l1 if l2, or
• One of the preconditions is always satisfied, or
  
• Observations (and obs. model) in 2-CNF

64
STRIPS-Filter Experimental Results
A. Russell 03
65
Tractability and Representation Size

• Theorem1 Every propositional repn. of the belief
  state grows exponentially for some systems, even
  when initial belief state is compactly
  represented (follows from Boppana Sipser 90)
• However, special cases can can be computed
  efficiently and represented compactly
• s in 2-CNF

1 Rough statement. Complete one in A. Russell
03.
66
STRIPS-Filter Experimental Results
A. Russell 03
67
Applications

• Tractable filtering and tracking of the world in
  high-dimensional domains with many objects,
  locations and relationships
• Learn effects and preconditions of actions in
  partially-observable domains
• Autonomous exploration of uncharted domains

68
Related Work

• Stochastic filtering
• Kalman 60, Blackman Popoli 99, Doucet
  et-al. 00,
• Action theories and semantics
• Gelfond Lifschitz 97, Baral Son 01,
  Doherty et-al. 98,
• Computation of progression
• Winslett 90, del Val 92, Lin Reiter
  97, Simon del Val 01,

69
THE END
70
Filtering STRIPS Actions

• STRIPS
• Action was executed or we observed an error,
• No conditional effects, and
• Possibly nondeterministic effects

71
Logical Filtering Progress Outlook

• 18 months relational filtering, learning actions
  in partially-observable domains
• 36 months dynamic observation models, Horn
  belief states, filtering in expanding worlds,
  autonomous agents in games
• 54 months first-order filtering, factored belief
  states, continuous time, autonomous exploration
  of uncharted domains

72
Today

• Probabilistic graphical models
• Treewidth methods
• Variable elimination
• Clique tree algorithm
• Applications du jour Sensor Networks

73
Contents

• Probabilistic graphical models
• Exact inference and treewidth
• Variable elimination
• Junction trees
• Applications du jour Sensor Networks

74
Application Planning

• General-purpose planning problem
• Given
• Domain features (fluents)
• Action descriptions effects, preconditions
• Initial state
• Goal condition
• Find
• Sequence of actions that is guaranteed to achieve
  the goal starting from the initial state

75
Application Planning with partitions

• PartPlan Algorithm
• Start with a tree-structured partition graph
• Identify goal partition
• Direct edges toward goal
• In each partition
• Generate all plans possible with depth d and
  width k
• Pass messages toward goal

76
Planning with partitions

• PartPlan Algorithm
• Start with a tree-structured partition graph
• Identify goal partition
• Direct edges toward goal
• In each partition
• Generate all plans possible with depth d and
  width k
• if you give me a block, I can return it to you
  painted,
• if you give me a block, let me do a few things,
  and then give me another block, then I can return
  the two painted and glued together.
• Pass messages toward goal
• All preconditions/effects for which there are
  feasible action sequences

77
Factored Planning Analysis

• Planner is sound and complete
• Running time for finding plans of width w with m
  partitions of treewidth k is O(mw22w2k)
• Factoring can be done in polynomial time
• Goal can be distributed over partitions by adding
  at most 2 features per partition

78
Next Time

• Probabilistic Graphical Models
• Directed models Bayesian Networks
• Undirected models Markov Fields
• Requires prior knowledge of
• Treewidth and graph algorithms
• Probability theory