CS 2710, ISSP 2160

1
CS 2710, ISSP 2160

• Chapter 9
• Inference in First-Order Logic

2
Pages to skim

• Storage and Retrieval (p. starts bottom 328)
• Efficient forward chaining (starts p. 333)
  through Irrelevant facts (ends top 337)
• Efficient implementation of logic programs
  (starts p. 340) through Constraint logic
  programming (ends p. 345)
• Completeness of resolution (starts p. 350)
  (though see notes in slides)

3
Inference with Quantifiers

• Universal Instantiation
• Given ?X (person(X) ? likes(X, sun))
• Infer person(john) ? likes(john,sun)
• Existential Instantiation
• Given ?x likes(x, chocolate)
• Infer likes(S1, chocolate)
• S1 is a Skolem Constant that is not found
  anywhere else in the KB and refers to (one of)
  the individuals that likes sun.

4
Reduction to Propositional Inference

• Simple form (pp. 324-325) not efficient. Useful
  conceptually.
• Replace each universally quantified sentence by
  all possible instantiations
• All X (man(X) ? mortal(X)) replaced by
• man(tom) ? mortal(tom)
• man(chocolate) ? mortal(chocolate)
• Now, we have propositional logic.
• Use propositional reasoning algorithms from Ch 7

5
Reduction to Propositional Inference

• Problem when the KB includes a function symbol,
  the set of term substitutions is infinite.
  father(father(father(tom)))
• Herbrand 1930 if a sentence is entailed by the
  original FO KB, then there is a proof using a
  finite subset of the propositionalized KB
• Since any subset has a maximum depth of nesting
  in terms, we can find the subset by generating
  all instantiations with constant symbols, then
  all with depth 1, and so on

6
First-Order Inference

• We have an approach to FO inference via
  propositionalization that is complete any
  entailed sentence can be proved
• Entailment for FOPC is semi-decidable algorithms
  exist that say yes to every entailed sentence,
  but no algorithm exists that also says no to
  every nonentailed sentence.
• Our proof procedure could go on and on,
  generating more and more deeply nested terms, but
  we will not know whether it is stuck in a loop,
  or whether the proof is just about to pop out

7
Generalized Modus Ponens

• This is a general inference rule for FOPC that
  does not require universal instantiation first
• Given
• p1, p2 pn, (p1 ? pn) ? q
• Subst(theta, pi) subst(theta, pi) for all i
• Conclude
• Subst(theta, q)

8
GMP is a lifted version of MP

• GMP lifts MP from propositional to first-order
  logic
• Key advantage of lifted inference rules over
  propositionalization is that they make only
  substitutions which are required to allow
  particular inferences to proceed

9
GMP Example

• ?x,y,z ((parent(x,y) ? parent(y,z)) ?
  grandparent(x,z))
• parent(james, john), parent(james, richard),
  parent(harry, james)
• We can derive
• Grandparent(harry, john), bindingsx/harry,y/jam
  es,z/john
• Grandparent(harry, richard), bindings
  x/harry,y/james,z/richard

10
Unification

• Process of finding all legal substitutions
• Key component of all FO inference algorithms
• Unify(p,q) theta, where Subst(theta,p)
  Subst(theta,q)
• Assuming all variables universally quantified

11
Standardizing apart

• All X knows(john,X).
• All X knows(X,elizabeth).
• These ought to unify, since john knows everyone,
  and everyone knows elizabeth.
• Rename variables to avoid such name clashes
  
• Note
• all X p(X) all Y p(Y)
• All X (p(X) q(X)) All X p(X) All Y q(Y)

12
def Unify (p, q, bdgs) d disagreement(p,
q) If there is no disagreement, then
success. if not d return bdgs elif not
isVar(d0) and not isVar(d1) return 'fail'
else if isVar(d0) var d0 other
d1 else var d1 other d0
if occursp (var,other) return fail
Make appropriate substitutions and recurse on
the result. else pp
replaceAll(var,other,p) qq
replaceAll(var,other,q) return Unify
(pp,qq, bdgs var,other)
For code, see resources on the webpage
13
unify
'loves', 'dog', 'var_x', 'dog', 'fred'
'loves', 'var_z', 'var_z' subs 'var_z',
'dog', 'var_x', 'var_x', 'fred' result
'loves', 'dog', 'fred', 'dog', 'fred'
unify
'loves', 'dog', 'fred', 'fred' 'loves',
'var_x', 'var_y' subs 'var_x', 'dog',
'fred', 'var_y', 'fred' result 'loves',
'dog', 'fred', 'fred'
unify 'loves', 'dog', 'fred',
'mary' 'loves', 'dog', 'var_x', 'var_y'
subs 'var_x', 'fred', 'var_y', 'mary'
result 'loves', 'dog', 'fred', 'mary'

14
unify 'loves', 'dog', 'fred', 'mary'
'loves', 'dog', 'var_x', 'var_y' subs
'var_x', 'fred', 'var_y', 'mary' result
'loves', 'dog', 'fred', 'mary'
unify
'loves', 'dog', 'fred', 'fred' 'loves',
'var_x', 'var_x' failure
unify 'loves', 'dog', 'fred',
'mary' 'loves', 'dog', 'var_x',
'var_x' failure
unify 'loves', 'var_x', 'fred'
'loves', 'dog', 'var_x', 'fred' var_x occurs
in 'dog', 'var_x' failure
15
unify 'loves', 'var_x', 'dog', 'var_x'
'loves', 'var_y', 'var_y' var_y occurs in
'dog', 'var_y' failure
unify 'loves', 'var_y', 'var_y'
'loves', 'var_x', 'dog', 'var_x' var_x
occurs in 'dog', 'var_x' failure
unify
(fails because vars not standardized apart)
'hates', 'agatha', 'var_x' 'hates',
'var_x', 'f1', 'var_x' failure
unify
'hates', 'agatha', 'var_x' 'hates',
'var_y', 'f1', 'var_y' subs 'var_y',
'agatha', 'var_x', 'f1', 'agatha' result
'hates', 'agatha', 'f1', 'agatha'
16
Most General Unifier

• The Unify algorithm returns a MGU
• L1 p(X,f(Y),b)
• L2 p(X,f(b),b)
• Subst1 X\a, Y\b
• Result1 p(a,f(b),b)
• Subst2 Y\b
• Result2 p(X,f(b),b)
• Subst1 is more restrictive than Subst2. In fact,
• Subst2 is a MGU of L1 and L2.

17
Storage and retrieval

• Hash statements by predicate for quick retrieval
  (predicate indexing), e.g., of all sentences that
  unify with tall(X)
• Why attempt to unify
• tall(X) and silly(dog(Y))
• Instead
• Predicatestall all tall facts
• Unify(tall(X),s) for s in Predicatestall
• Subsumption lattice for efficiency (see p. 329
  for your interest)

18
Inference Methods

• Unification (prerequisite)
• Forward Chaining
• Production Systems
• RETE Method (OPS)
• Backward Chaining
• Logic Programming (Prolog)
• Resolution
• Transform to CNF
• Generalization of Prop. Logic resolution

19
Resolution Theorem Proving (FOL)

• Convert everything to CNF
• Resolve, with unification
• Save bindings as you go!
• If resolution is successful, proof succeeds
• If there was a variable in the item to prove,
  return variables value from unification bindings

20
Converting to CNF
21
Converting sentences to CNF

• 1. Eliminate all ? connectives
• (P ? Q) ? ((P ? Q) (Q ? P))
• 2. Eliminate all ? connectives
• (P ? Q) ? (?P ? Q)
• 3. Reduce the scope of each negation symbol to a
  single predicate
• ??P ? P
• ?(P ? Q) ? ?P ? ?Q
• ?(P ? Q) ? ?P ? ?Q
• ?(?x)P ? (?x)?P
• ?(?x)P ? (?x)?P
• 4. Standardize variables rename all variables so
  that each quantifier has its own unique variable
  name

22
Converting sentences to clausal form Skolem
constants and functions

• 5. Eliminate existential quantification by
  introducing Skolem constants/functions
• (?x)P(x) ? P(c)
• c is a Skolem constant (a brand-new constant
  symbol that is not used in any other sentence)
• (?x)(?y)P(x,y) becomes (?x)P(x, F(x))
• since ? is within the scope of a universally
  quantified variable, use a Skolem function F to
  construct a new value that depends on the
  universally quantified variable
• f must be a brand-new function name not occurring
  in any other sentence in the KB.
• E.g., (?x)(?y)loves(x,y) becomes
  (?x)loves(x,F(x))
• In this case, F(x) specifies the person that x
  loves
• E.g., ?x1 ?x2 ?x3 ?y P( y ) becomes
• ?x1 ?x2 ?x3 P( FF(x1,x2,x3) ) (FF is a new
  name)

23
Converting sentences to clausal form

• 6. Remove universal quantifiers by (1) moving
  them all to the left end (2) making the scope of
  each the entire sentence and (3) dropping the
  prefix part
• Ex (?x)P(x) ? P(x)
• 7. Put into conjunctive normal form (conjunction
  of disjunctions) using distributive and
  associative laws
• (P ? Q) ? R ? (P ? R) ? (Q ? R)
• (P ? Q) ? R ? (P ? Q ? R)
• 8. Split conjuncts into separate clauses
• 9. Standardize variables so each clause contains
  only variable names that do not occur in any
  other clause

24
An example

• (?x)(P(x) ? ((?y)(P(y) ? P(F(x,y))) ?
  ?(?y)(Q(x,y) ? P(y))))
• 2. Eliminate ?
• (?x)(?P(x) ? ((?y)(?P(y) ? P(F(x,y))) ?
  ?(?y)(?Q(x,y) ? P(y))))
• 3. Reduce scope of negation
• (?x)(?P(x) ? ((?y)(?P(y) ? P(F(x,y)))
  ?(?y)(Q(x,y) ? ?P(y))))
• 4. Standardize variables
• (?x)(?P(x) ? ((?y)(?P(y) ? P(F(x,y)))
  ?(?z)(Q(x,z) ? ?P(z))))
• 5. Eliminate existential quantification
• (?x)(?P(x) ?((?y)(?P(y) ? P(F(x,y))) ?(Q(x,G(x))
  ? ?P(G(x)))))
• 6. Drop universal quantification symbols
• (?P(x) ? ((?P(y) ? P(F(x,y))) ?(Q(x,G(x)) ?
  ?P(G(x)))))

25
An Example

• 7. Convert to conjunction of disjunctions
• (?P(x) ? ?P(y) ? P(F(x,y))) ? (?P(x) ? Q(x,G(x)))
  ?
• (?P(x) ? ?P(G(x)))
• 8. Create separate clauses
• ?P(x) ? ?P(y) ? P(F(x,y))
• ?P(x) ? Q(x,G(x))
• ?P(x) ? ?P(G(x))
• 9. Standardize variables
• ?P(x) ? ?P(y) ? P(F(x,y))
• ?P(z) ? Q(z,G(z))
• ?P(w) ? ?P(G(w))
• Note Now that quantifiers are gone, we do need
  the upper/lower-case distinction

26
1. all X (read (X) --gt literate (X)) 2. all X
(dolphin (X) --gt literate (X)) 3. exists X
(dolphin (X) intelligent (X)) (a translation of
Some dolphins are intelligent'') Are there
some who are intelligent but cannot read?'' 4.
exists X (intelligent(X) read (X)) Set of
clauses (1-3) 1. read(X) v literate(X) 2.
dolphin(Y) v literate(Y) 3a. dolphin (a) 3b.
intelligent (a) Negation of 4 (exists Z
(intelligent(Z) read (Z))) In Clausal
form intelligent(Z) v read(Z) Resolution proof
in lecture.
27
More complicated example Did Curiosity kill the
cat

• Jack owns a dog. Every dog owner is an animal
  lover. No animal lover kills an animal. Either
  Jack or Curiosity killed the cat, who is named
  Tuna. Did Curiosity kill the cat?
• These can be represented as follows
• A. (?x) (Dog(x) ? Owns(Jack,x))
• B. (?x) (((?y) (Dog(y) ? Owns(x, y))) ?
  AnimalLover(x))
• C. (?x) (AnimalLover(x) ? ((?y) Animal(y) ?
  ?Kills(x,y)))
• D. Kills(Jack,Tuna) ? Kills(Curiosity,Tuna)
• E. Cat(Tuna)
• F. (?x) (Cat(x) ? Animal(x) )
• G. Kills(Curiosity, Tuna)

GOAL
28

• Convert to clause form
• A1. (Dog(D))
• A2. (Owns(Jack,D))
• B. (?Dog(y), ?Owns(x, y), AnimalLover(x))
• C. (?AnimalLover(a), ?Animal(b), ?Kills(a,b))
• D. (Kills(Jack,Tuna), Kills(Curiosity,Tuna))
• E. Cat(Tuna)
• F. (?Cat(z), Animal(z))
• Add the negation of query
• ?G (?Kills(Curiosity, Tuna))

D is a skolem constant
29

• The resolution refutation proof
• R1 ?G, D, (Kills(Jack, Tuna))
• R2 R1, C, a/Jack, b/Tuna (AnimalLover(Jack),
  Animal(Tuna))
• R3 R2, B, x/Jack (Dog(y), Owns(Jack, y),
  Animal(Tuna))
• R4 R3, A1, y/D (Owns(Jack, D),
  Animal(Tuna))
• R5 R4, A2, (Animal(Tuna))
• R6 R5, F, z/Tuna (Cat(Tuna))
• R7 R6, E, FALSE

30

• The proof tree

?G
D

C
R1 K(J,T)
a/J,b/T
B
R2 ?AL(J) ? ?A(T)
x/J
A1
R3 ?D(y) ? ?O(J,y) ? ?A(T)
y/D
A2
R4 ?O(J,D), ?A(T)

F
R5 ?A(T)
z/T
A
R6 ?C(T)

R7 FALSE
31
Decidability and Completeness

• Resolution is a refutation complete inference
  procedure for First-Order Logic
• If a set of sentences contains a contradiction,
  then a finite sequence of resolutions will prove
  this.
• If not, resolution may loop forever
  (semi-decidable)
• Here are notes by Charles Elkan that go into this
  more deeply

32
Decidability and Completeness

• Refutation Completeness If KB A then KB - A
• If its entailed, then theres a proof
• Semi-decidable
• If theres a proof, well halt with it.
• If not, maybe halt, maybe not
• Logical entailment in FOL is semi-decidable if
  the desired conclusion follows from the premises,
  then eventually resolution refutation will find a
  contradiction

33
Decidability and Completeness

• Propositional logic
• logical entailment is decidable
• There exists a complete inference procedure
• First-Order logic
• logical entailment is semi-decidable
• Resolution procedure is refutation complete

34

• Strategies (heuristics) for efficient resolution
  include
• Unit preference. If a clause has only one
  literal, use it first.
• Set of support. Identify useful rules and
  ignore the rest. (p. 305)
• Input resolution. Intermediately generated
  sentences can only be combined with original
  inputs or original rules.
• Subsumption. Prune unnecessary facts from the
  database.

35
Horn Clauses

• A Horn Clause is a CNF clause with at most one
  positive literal
• Horn Clauses form the basis of forward and
  backward chaining
• The Prolog language is based on Horn Clauses
• Deciding entailment with Horn Clauses is linear
  in the size of the knowledge base

36
Reasoning with Horn Clauses

• Forward Chaining
• For each new piece of data, generate all new
  facts, until the desired fact is generated
• Data-directed reasoning
• Backward Chaining
• To prove the goal, find a clause that contains
  the goal as its head, and prove the body
  recursively
• Goal-directed reasoning
• The state space is an AND-OR graph see 7.5.4

37
Forward Chaining over FO Definite (Horn) Clauses

• Clauses (disjunctions) with at most one positive
  literal
• First-order literals can include variables, which
  are assumed to be universally quantified
• Use GMP to perform forward chaining
• (Semi-decidable as for full FOPC)

38
Def FOL-FC-Ask(KB,A) returns subst or false
KB set of FO definite clauses with variables
standardized apart A the query, an atomic
sentence Repeat until new is empty new ?
for each implication (p1 pn ? q) in KB
for each T such that SUBST(T,p1pn)
SUBST(T,p1pn) for some p1,,pn in KB
q ? SUBST(T,q) if q is not
a renaming of a sentence already in KB or new
add q to new S ?
Unify(q,A) if S is not fail
then return S add new to KB Return false
Process can be made more efficient read on your
own, for interest
39
Backward Chaining over Definite (Horn) Clauses

• Logic programming
• Prolog is most popular form
• Depth-first search, so space requirements are
  lower, but suffers from problems from repeated
  states

40
american(X) weapon(Y) sells(X,Y,Z)
hostile(Z) ? criminal(X). owns(nono,m1).
missile(m1). missile(X1) owns(nono,X1) ?
sells(west,X1,nono). missile(X2) ?
weapon(X2). enemy(X3,america) ?
hostile(X3). american(west). enemy(nono,america).
Goal criminal(west). Backward chaining
proof in lecture In Prolog criminal(X) -
american(X), weapon(Y), sells(X,Y,Z), hostile(Z).

41
Horn clauses are all of the form L1 L2
... Ln -gt Ln1 Or, equivalently, in clausal
form L1 v L2 v ... v Ln v Ln1 Prolog
(like databases) makes the "closed world
assumption" if P cannot be proved, infer not
P Think of the system as an arrogant
know-it-all "If it were true, I would know it.
Since I can't prove it, it must not be
true" Thus, it uses "negation as failure".
42
neighbor(canada,us) neighbor(mexico,us) neighbor(
pakistan,india) ?- neighbor(canada,india).
no In full first-order logic, you would have
to be able to infer neighbor(canada,india)"
for "neighbor(canada,india)" to be false. Be
careful! neighbor(canada,india) is not
entailed by the Sentences above!
43
bachelor(X) - male(X), \ married(X). male(bill).
male(jim). married(bill). married(mary). An
individual is a bachelor if it is male and it
is not married. \ is the negation-as-failure
operator in Prolog. ?- bachelor(bill). no
?- bachelor(jim). yes ?- bachelor(mary). no
?- bachelor(X). X jim no ?-
44
Comparing backward chaining in prolog with
resolution

• In lecture

45
WrapUp

• You are responsible for everything in Chapter 9
  except the following (though you are encouraged
  to read them)
• Storage and Retrieval (p. starts bottom 328)
• Efficient forward chaining (starts p. 333)
  through Irrelevant facts (ends top 337)
• Efficient implementation of logic programs
  (starts p. 340) through Constraint logic
  programming (ends p. 345)
• Completeness of resolution (starts p. 350)
  (though see notes in slides)
• Also, see files posted on the schedule (clausal
  form conversion, resolution, etc.)