Advanced Artificial Intelligence Lecture 20: Olsson: Inductive Functional Programming

1
Advanced Artificial IntelligenceLecture 20
Olsson Inductive FunctionalProgramming

• Bob McKay
• School of Computer Science and Engineering
• College of Engineering
• Seoul National University

2
Outline

• ADATE Program synthesis system

3
Roland Olsson

• Inductive Functional Programming using
  Incremental Program Transformation
• Artificial Intelligence 74 (1995), 55-81
• Actually, just one of a series of papers on
  program transformation and program learning
• System called ADATE
• Automatic Design of Algorithms Through Evolution
• Despite the name, not an evolutionary algorithm
• _very_ deterministic search methods
• Many versions

4
Inductive Functional Programming

• Representation
• Clean fragment of a standard functional
  programming language
• Standard ML
• Search
• Deterministic
• Iterative deepening
• Operators
• The main focus of Olssons work

5
Problem Specification

• A critique of ILP-style approaches
• Problem specification as set of input/output
  pairs
• Correct or incorrect
• Input/output pairs
• Usually only one output per input
• A functional, rather than relational,
  specification
• Unfairly biases search
• Argues for real-valued evaluations (not just 0/1)
• Similar to GP fitness functions
• Interesting because of time
• PhD 1994 - early GP
• More based on ILP than GP

6
Program Evaluation (Fitness) Function

• Components of Fitness
• Syntactic complexity
• Let mi be the number of alternatives at node i of
  expression tree
• S ?i log mi (that is, total information
  content)
• T - Time complexity
• Nc - Number of Correct Outputs
• Note programs may give dont know outputs
• Nw - Number of Wrong Outputs
• Overall fitness functions difficult to understand

7
Language

• Standard ML with omissions
• Currying
• write f(x,y) rather than (f(x))(y)
• Boolean operators
• If expressions
• References

8
Transformations Abstraction

• Given a subexpression
• H(E1,, En)
• In a program, replace it with
• Let fun f(V1,, Vn) H(V1,, Vn) in g(E1,, En)
  end
• That is, introduce a new function definition
  equal to that subexpression

9
Transformations Case Distribution

• Case distribution distributes function
  applications over cases
• If we have
• h(A1,,Ai, case E of Match1 gt E1
  Matchn gt En Ai2,,Am)
• Replace it with
• case E of Match1 gt h(A1,,Ai, E1, Ai2,,Am)
  Matchn gt h(A1,,Ai, En, Ai2,,Am)
• Or vice-versa
• That is, the transformation can be used in either
  direction

10
Transformations Embedding

• Given a let definition
• Let fun f(V1,, Vn) RHS in Exp end
• Embedding replaces the type expression for one of
  the Vi with a compound type (list, tree or
  similar)
• This means that the types in RHS and Exp must
  change also
• There are further special restrictions to ensure
  consistency and avoid potentially long (or even
  infinite) recursions of type replacement

11
TransformationsExpression Synthesis

• ADATE synthesises all epxressions of depth 1,
  then of depth 2,.
• It uses heuristics to prune the search space of
  all possible syntheses
• For synthesised case-expressions, before fully
  synthesising, ADATE executes the partial program
  to test that more than one case is activated if
  not, the case is omitted
• For synthesised recursions, ADATE uses a partial
  ordering on expressions to ensure that the RHS
  case is in some sense simpler than that on the LHS

12
Compound Transformations

• ADATE synthesises programs starting with a
  trivial program and applying sequences of
  transformations
• A compound transformation
• The sequence of transformations is highly
  restricted
• Only particular transformations may come after
  others

13
ADATE search

• ADATE search is a complex deterministic search
  method
• It searches by smallest number of case
  expressions
• For a given number of case expressions, it
  maintains three programs, the best so far using
  each of three fitness measures, which favour
• Correctness
• Size
• Time complexity

14
Analysis of Recursion

• Interesting point that recursion generally
  requires coupled development of two expressions
• General case
• Terminating case
• Fitness only improves if they are properly linked
• Difficult for GP to do
• Not explainable by schema theory

15
Analysis of Recursion

• Possible ways of overcoming this
• Add operators that couple changes to recursive
  case and terminating case
• Change representation so that recursive case and
  terminating case are linked

16
ADATE More Recent Work

• Population Management for Automatic Design of
  Algorithms through Evolution (1998)
• Despite the title, deterministic search
• But discusses some issues of population control
• Relevant to beam search as well as to
  evolutionary search

17
ADATE More Recent Work

• How to invent functions (1999)
• More detailed study of abstraction
• Especially examination of heuristics for choosing
  suitable abstractions
• Argues that ADATE is a form of evolutionary
  computation, though not stochastic
• Im not sure that this is a useful categorisation

18
ADATE More Recent Work

• Self-Improvement for the ADATE Automatic
  Programming System (2002)
• Uses ADATE to generate transformations of its own
  search space
• Does not result in significant improvement

19
ADATE Evaluation

• Interesting performance
• Generally comparable with complexity of ILP
  programs
• Seems to handle recursion well
• Some very tough problems solved
• Binary Search Tree deletion, insertion
• Deterministic search
• But very complex
• Can be considered a form of tightly-controlled
  beam-search

20
ADATE Evaluation

• Results were highly impressive for 1995
• There doesnt seem to have been much progress
  since
• Not clear that this is a productive research
  direction
• But transformations may be useful in more
  stochastic searches

21
Summary

• ADATE
• Deterministic algorithm
• Complex beam search
• Shares many characteristics with evolutionary
  search
• But not stochasticity

22
?????