Advanced Artificial Intelligence Lecture 7: Language

1
Advanced Artificial IntelligenceLecture 7
Language Search BiasRDT Grendel

• Bob McKay
• School of Computer Science and Engineering
• College of Engineering
• Seoul National University

2
Outline

• Declarative Specification of Bias
• Mobal/RDT
• Grendel

3
Mobal/RDT (Kietz Wrobel, 1991)

• Language function-free Horn clauses
• Mobal
• RDT is one component of a major project, Mobal
• Mobal is described as 'an environment for
  incremental modelling'
• Perhaps it is easiest to think of it as an
  environment for acquiring knowledge for expert
  systems
• As such, Mobal provides many integrated tools for
  acquiring and regularising knowledge
• RDT is the tool provided for acquiring relational
  knowledge from data
• Thus RDT has a more pragmatic approach than
  previously discussed systems, even FOIL

4
Specification of Hypothesis Language

• The major contribution of RDT is in its use of
  declaratively specified restrictions of the
  hypothesis language
• Rule models
• Rule models are restricted second order formulae
  they consist of rules in which the predicate
  symbols may be variables
• The first order language searched is the set of
  subclauses of substition instances of the rule
  models
• For example, suppose the rule model includedP(Y)
  R(X,Y) ? c(X)
• Mobal would seek instantiations of P and R (and
  possibly of X or Y) which would yield accurate
  rules with high coverage

5
Specification of Hypothesis Language

• The major contribution of RDT is in its use of
  declaratively specified restrictions of the
  hypothesis language
• Rule models
• Rule models are restricted second order formulae
  they consist of rules in which the predicate
  symbols may be variables
• The first order language searched is the set of
  subclauses of substition instances of the rule
  models
• For example, suppose the rule model includedP(Y)
  R(X,Y) ? c(X)
• Mobal would seek instantiations of P and R (and
  possibly of X or Y) which would yield accurate
  rules with high coverage

6
Search Mechanism

• RDT uses a breadth first, general-to-specific
  search of the specified language
• It starts with the most general rule models it
  has available
• It then searches specialisations of this rule
  model
• For example, suppose the rule modelP(Y) R(X,Y)
  ? c(X)has already been searched and found to be
  too general
• RDT will then search for any rule models that are
  more specific
• For example, P(Y) Q(Y) R(X,Y) ? c(X)
• Hypotheses that have already been either accepted
  as valid or pruned as too special are remembered
• New candidate hypotheses are first checked to see
  if they are specialisations of remembered
  hypotheses
• In this case, there is no point in testing further

7
Heuristics

• Instead of FOIL's information theoretic
  heuristics, RDT provides simple user-settable
  heuristics
• The user may specify simple arithmetic functions
  of system variables
• The number of instances correctly predicted by
  the hypothesis
• The number incorrectly predicted by the
  hypothesis to be members of the class
• The number incorrectly predicted not to be
  members of the class
• etc
• The formula so specified will be used as the
  heuristic to guide search

8
Higher Order Generalisation

• The main complexity in RDT stems from the use of
  second order variables
• In first order logic, the relationship between
  variable substitution and generality is simple
• A substitution instance of a formula is always
  less general than the original formula
• For example, if we apply a substition that takes
  Y to X, to a formula p(X,Y), we get p(X,X), which
  is more specific than the original
• This is not the case in higher order logics
• If we apply a substitution that takes P2 to P1 to
  the formula P1(X,Y) P2(X,Y), we get the formula
  P1(X,Y), which is more general than the original

9
Sequence of Instantiations

• A somewhat related problem concerns the order in
  which the predicate variables within a rule model
  should be instantiated
• With a model such asP(Y) R(X,Y) ? c(X)It
  makes little sense to instantiate P first
• Variable Y in the hypothesis is unrelated to
  variable X in the conclusion
• The rule would not be predictive

10
Variable Connection

• RDT handles this with the idea of the connection
  of a hypothesis variable to the conclusion
• A literal is directly connected to another
  literal if they share a variable
• Predicate variables are substituted in the order
  determined by these connection chains
• A predicate variable may only be substituted if
  it is directly connected to a literal which
  already has an instantiated predicate symbol

11
Background Knowledge

• Background knowledge consists of rules which are
  already known before learning starts
• RDT uses background knowledge by transforming it
  to a set of ground facts
• To ensure this can be done, background knowledge
  is constrained to consist of syntactically
  generative rules

12
Syntactically Generative

• A clause A1 .... Am ? Bis syntactically
  generative if the varables in B are a subset of
  the variables in A1,...., Am
• For examplemember(X,Ys) ? member(X,YYs)is
  not syntactically generative
• Y occurs in the conclusion but not the antecedent
  of the rule
• member(X,Ys) integer(Y) ? member(X,YYs)is
  syntactically generative
• A non-generative clause can always be made
  generative by adding typing or other information

13
RDT Search Restrictions

• Typing
• RDT is strongly typed, and uses this typing to
  restrict the search spaces
• Only type-consistent hypotheses are examined
• Topologies
• RDT also allows the user to specify rule concept
  'topologies'
• A topology specifies a dependency relationship
  between predicates
• The search is constrined to be consistent with
  the topology

14
Grendel (W Cohen, 1993)

• Hypothesis language function free Horn clauses.
• Grendel continues some of the ideas of RDT,
  providing mechanisms for the user to
  declaratively specify bias
• Grendel uses grammars instead of higher order
  logic
• Grendel implementation
• Grendel is built on top of FOIL

15
Grendel and Declarative Specification of Language
Bias

• Grendel users define the search space by use of
  an Antecedent Description Grammar (ADG)
• An ADG is essentially a context free grammar
  (CFG) in which the symbols are logical literals (
• ADGs are closely related to prolog Definite
  Clause Grammars
• They may be expressed in higher-order logic
  programming languages as higher-order DCGs
• The basic idea is that the only hypotheses
  considered are those which may be generated by
  the ADG
• In this form, Grendel provides only language bias
  
• Cohen was able to demonstrate that virtually any
  language bias found in the literature could be
  expressed in this form

16
ADG Example

• goal_formula(illegal(A,B,C,D,E,F)).
• body(illegal(A,B,C,D,E,F)) ? rels(A,B,C,D,E,F).
• rels(A,B,C,D,E,F) ?.rels(A,B,C,D,E,F) ?
  rel(A,B,C,D,E,F), rels(A,B,C,D,E,F).
• rel(A,B,C,D,E,F) ? pred(X,Y) where
  member(X,A,B,C,D,E,F), member(Y,A,B,C,D,E,F).
• pred(X,Y) ?X Y.pred(X,Y) ?not X
  Y.pred(X,Y) ?adj(X,Y).pred(X,Y) ?not
  adj(X,Y).pred(X,Y) ? less_than(X,Y).pred(X,Y)
  ?not less_than(X,Y).

17
Search Bias in Grendel

• Search bias is represented in Grendel by the use
  of deferred grammar rules
• A grammar rule is deferred if it is prefixed with
  an exclamation mark
• For example, Grendel might be informed to prefer
  positive literals by replacing the productions
  containing negative literals in our previous
  example with

18
Search Bias Example

• goal_formula(illegal(A,B,C,D,E,F)).
• body(illegal(A,B,C,D,E,F)) -gt rels(A,B,C,D,E,F).
• rels(A,B,C,D,E,F) -gt .rels(A,B,C,D,E,F) -gt
  rel(A,B,C,D,E,F), rels(A,B,C,D,E,F).
• rel(A,B,C,D,E,F) -gt pred(X,Y) where
  member(X,A,B,C,D,E,F), member(Y,A,B,C,D,E,F).
• pred(X,Y) -gt X Y.!pred(X,Y) -gt not X
  Y.pred(X,Y) -gt adj(X,Y).!pred(X,Y) -gt not
  adj(X,Y).pred(X,Y) -gt less_than(X,Y).!pred(X,
  Y) -gt not less_than(X,Y).

19
Deferred Rules

• Grendel uses deferred rules to control its search
  order
• Learning is first done using only the
  non-deferred part of the grammar
• The deferred part of the grammar is only used
  when Grendel fails to find a literal with
  positive information gain
• Once a literal with positive information gain has
  been found, the algorithm returns to using only
  the non-deferred portion of the grammar
• This mechanism is extremely simple, and more
  complex ones are readily devised

20
Summary

• Declarative Specification of Bias
• Mobal/RDT
• Specifying Search Space
• Higher Order Models
• Type Restrictions
• Topology Restrictions
• Grendel
• Specifying Search Space
• Grammars
• Specifying Search Order
• Multi-level Grammars

21
?????