Advanced Artificial Intelligence Relational Learning RLGGs

1
Advanced Artificial IntelligenceRelational
LearningRLGGs Golem

• Bob McKay
• School of Computer Science and Engineering
• College of Engineering
• Seoul National University

2
Resolution and Clauses

• Recall that L1 Ln ? L0 and L0 Ln ?
  L1 are logically equivalent
• So we can define a (Horn) clause as a set of
  literals L1, , Ln , of which at least one is
  negative
• The resolvent with respect to L of clauses L,
  M1, , Mm and L, N1, , Nn is the clause
  M1, , Mm , N1, , Nn
• Resolution of two clauses consists of finding
  complementary literals in them, and joining them
  together while deleting the complementary
  literals
• The resolvent of two clauses is implied by the
  combination of the original two clauses

3
Least General Generalisation

• One way for one clause to imply another is for
  the first to contain all the literals of
  (subsume) the second
• Another way is for the second to be a
  substitution instance (via a substitution ?) of
  the first
• Combining these, we get the definition of
  ?-subsumption C ?-subsumes D iff there is a
  substitution ? such that ? applied to C gives a
  subset of D
• ?-subsumption generates a generalisation
  hierarchy
• The Least General Generalisation (LGG) of a set
  of clauses S is the least location in the
  ?-subsumption hierarchy that ?-subsumes each of S

4
Relative Least General Generalisation

• An alternative approach derives from Plotkins
  very early work (1970) on logic-based learning
• Plotkin defined the Least General Generalisation
  we saw previously
• However the LGG can't handle background knowledge
• Requires an extension to LGG relative to
  background knowledge
• C generalises D relative to theory T iff C
  ?-subsumes D together with the negations of the
  literals in T
• Plotkins idea was to simply compute the RLGG of
  a set of instances, and treat that as the learnt
  knowledge
• However it turned out to be impractical due to
  computational problems

5
Computational Problems with RLGG

• Construction of the RLGG requires computation of
  a ground model of the theory
• often no finite ground model exists
• Plotkin's algorithm leads to highly redundant
  RLGGs
• Depending on the theory, reduction of the
  redundancy is
• at best exponential time
• at worst undecidable
• Even the reduced RLGG may be infinite
• Even if we use a finite subset of the ground
  model
• which means the true RLGG is not being obtained
• the RLGG is usually exponential in the number of
  examples

6
RLGGs Revisited Golem(1992)

• Muggleton and Feng proposed a number of
  heuristics and restrictions aiming to avoid the
  problems encountered by Plotkin, generating the
  Golem system
• Restrictions on
• The hypothesis language
• The allowable forms of background knowledge
• Heuristics for
• A new approach to logical reduction

7
Golem Background Knowledge

• Golem restricts the background knowledge to be
  syntactically generative
• A1 .... Am gt B
• is syntactically generative if the variables in B
  are a subset of the variables in A1,...., Am
• Not syntactically generative
• member(X,Ys) ? member(X,YYs)
• Syntactically generative
• member(X,Ys) integer(Y) ? member(X,YYs)

8
Golem Finite Ground Model

• An atom A of a theory K is h-easy if A can be
  derived from K in at most h applications of
  resolution
• They then define Mh(K) to be the Herbrand
  instantiations of h-easy atoms of K (the
  Herbrand instantiations can be thought of as the
  instantiations that have to exist just because of
  the symbols in the language)
• Mh(K) is thus a finite subset of a ground model
  of K (in fact, its at most exponential in K)

9
Golem Restricted Hypothesis Language

• Recall the idea of determinacy from FOIL
• a new variable is determinate if its value is
  uniquely determined by the values of the
  pre-existing variables.
• Muggleton Feng extended this to the concept of
  ij-determinacy
• Effectively, measures the complexity of
  dependency chains
• If we bound i and j, the size of the RLGG of n
  examples is no longer exponential in n
• In fact, the maximum size is no longer dependent
  on n at all

10
Golem Clause Reduction

• Golem uses two methods to logically reduce
  clauses
• Functional Reduction
• Negative-based Reduction
• So far, we have discussed learning only in the
  presence of positive examples of a concept
• Golem also uses negative examples (ie possible
  tuples that are not part of the concept)
• These are held in a negative example file
• Any literal may be removed from the RLGG
• So long as its removal does not allow the
  derivation of a negative example

11
Golem Functional Reduction

• Golem uses mode declarations, specifying which
  variables of a predicate will be used for input
  values, and which for output
• From these declarations, Golem can construct a
  functional graph showing the dependency of
  predicates in a clause
• Given a candidate RLGG, Golem attempts to
  construct a functional graph of the hypothesis
  using only literals within the RLGG
• If it succeeds, any literals not occurring in the
  functional graph are deleted from the RLGG
• Note that functional reduction relies on the
  assumption of determinacy

12
?????