Advanced Artificial Intelligence Lecture 17: Evolutionary Algorithms

1
Advanced Artificial IntelligenceLecture 17
Evolutionary Algorithms

• Bob McKay
• School of Computer Science and Engineering
• College of Engineering
• Seoul National University

2
Outline

• Evolutionary Algorithms as stochastic search
  algorithms
• Biological Inspirations of Evolutionary
  Algorithms
• Variants of Evolutionary Algorithms
• Applications of Evolutionary Algorithms
• Estimation of Distribution Algorithms

3
Where were Going

• The rest of this course will look at evolutionary
  approaches to the problem of learning complex
  knowledge
• Genetic Programming and related methods tackle
  similar problems to ILP, RL, and SRL
• There hasnt been much work on integrating
  GP-like approaches and ILP/RL approaches
• There hasnt been much work on stochastic
  representation in GP-like approaches
• There is plenty of opportunity for
  cross-fertilisation
• But first, what are evolutionary algorithms?

4
Reminder Complete Search

• You have previously studied complete search
  algorithms
• Breadth-First Search
• Depth-First, Backtracking Search
• The former is guaranteed to find a solution, but
  maybe very inefficiently
• The latter may be faster, but can also get into
  infinite loops
• Neither takes advantage of any knowledge about
  the problem space

5
Reminder Heuristic Search

• Heuristic search algorithms extend complete
  search algorithms with a heuristic to guid search
• Heuristic in this context, a value indicating
  how good a potential solution is
• Deterministic Hillclimbing Algorithm
• CandInitial
• While not(solution(Cand)) do
• Newnext_neighbour(Cand)
• if heur(New) gt heur(Cand) Cand New
• End while

6
Deterministic Hillclimbing Search

• Deterministic hillclimbing search can be very
  fast and effective in simple search spaces
• But it has a serious problem
• If the search starts close to a local optimum, it
  will get stuck at the local optimum
• If a search landscape has many local optima,
  deterministic hillclimbing will perform poorly

7
Stochastic Hillclimbing Search

• A small change to deterministic hillclimbing
  makes it a much more reliable algorithm
• Introduce a small probability ?, and accept a new
  solution even if it is worse, with probability ?
• CandInitial
• While not(solution(Cand)) do
• Newnext_neighbour(Cand)
• random(P)
• if (heur(New) gt heur(Cand) or (P lt ?)) Cand
  New
• End while

8
Stochastic Hillclimbing Search

• However such an algorithm can still get stuck in
  infinite loops depending on the next_neighbour
  algorithm
• Generating the neighbour randomly can help
• CandInitial
• While not(solution(Cand)) do
• Newrandom_mutate(Cand)
• random(P)
• if (heur(New) gt heur(Cand) or (P lt ?)) Cand
  New
• End while

9
Stochastic Hillclimbing Search

• Considering the different possible gene alleles
  (values) as states, the search algorithm has
  become a Markov chain
• In particular, with most probability choices, it
  satisfies the ergodic property
• As a consequence, the probability of eventually
  finding the optimal solution is 1.0 for most
  variants
• Of course, eventually might be a very long time

10
Multi-start Hillclimbing Search

• Our initial starting point might be very bad
• We can run the algorithm multiple times
• Repeat
• Candrandom
• While not termination_condition do
• Newrandom_mutate(Cand)
• random(P)
• if (heur(New) gt heur(Cand) or (P gt ?)) Cand
  New
• End while
• Until solution(Cand)

11
Parallel Hillclimbing Search

• But it might be more efficient to do it in
  parallel
• Poprandom
• While not (solution ? Pop do
• randomly select Ind ? Pop biased by heur(Ind)
• Newrandom_mutate(Ind)
• if (heur(New) gt heur(Ind)) replace Ind by New in
  Pop
• End while

12
Evolution Strategies

• This is a common form of an evolutionary
  algorithm known as an evolution strategy
• Most commonly, in evolution strategies, an
  individual is a vector of real numbers, and the
  mutation operation is Gaussian mutation
• The individual New is generated by setting the
  mean of the Gaussian (normal) distribution to
  Ind, and the standard deviation ? to some fixed
  value, then sampling to find New
• More sophisticated versions either change ?
  according to a fixed schedule, or evolve ? as
  well
• Enables the algorithm to find the optimum
  accurately

13
Generational Hillclimbing Search

• A variant uses separate generations
• Pop(0)random t0
• While not (solution ? Pop(t)) do
• while size(Pop(t1)) lt size(Pop(t)) do
• randomly select Ind ? Pop(t) biased by
  heur(Ind)
• Newrandom_mutate(Ind)
• add New to Pop(t1)
• end while
• tt1
• End while

14
Recombination in Hillclimbing Search

• So far, the only way we have used to generate new
  individuals is to change existing ones in some
  stochastic way
• Mutation
• But for many problems, combining good candidate
  solutions is another good way to generate new
  good candidates

15
Recombinational Hillclimbing Search

• Pop(0)random t0
• While not (solution ? Pop(t)) do
• while size(Pop(t1)) lt size(Pop(t)) do
• randomly select Ind1, Ind2 ? Pop(t)
• with probability p1
• Newrandom_mutate(Ind1)
• else with probability p2
• Newrandom_combine(Ind1, Ind2)
• else New Ind1
• add New to Pop(t1)
• end while
• tt1
• End while

16
Genetic Algorithm

• This algorithm is generally known as a genetic
  algorithm
• It is often used with a discrete representation
  (integer, string, binary)
• A wide range of stochastic mutation and
  recombination (crossover) operators can be
  defined
• Also a wide range of selection operators

17
Evolution
18
Darwinian Evolution (1850s)
Evolution

• Multiple populations competing for limited
  resources
• Dynamically changing populations with births and
  deaths
• Inheritance children are like their parents
• Variation children are not exactly the same as
  their parents
• Fitness different individuals have different
  probabilities to survive and reproduce

19
Geological Time Line of the World

• Mi
• l
• l
• ions
• of
• Years

20
Algorithmic Time Line of the World

• Mi
• l
• l
• ions
• of
• Years

21
Artificial Evolution (1950s)
Evolution

• Apply simple models of evolution to populations
  in a computer
• This might help us to
• understand evolution
• evolve useful things

22
A Generic Evolutionary Algorithm
Evolution

• Initialise a population of individuals
• Repeat
• Evaluate the fitness of the individuals for the
  given problem
• Repeat
• Probabilistically select parents according to
  their fitness
• Apply sources of variation to parents to generate
  children
• Until there are sufficient children for the next
  population
• Until the problem is solved, evolution stagnates,
  or resources are exhausted

23
Variants ofStandardEvolutionary Algorithms
24
Generation Structure
Variants

• Steady state
• Children compete with parents
• Evolution Strategies
• Evolutionary Programming
• Kangaroos

• Generational
• Children replace parents
• Genetic Algorithms
• Genetic Programming
• Antechinus
• Many insects

25
Operators Mutation only
Variants

• Early Evolution Strategies
• Early Evolutionary Programming
• Bacteria

26
Mutation and Crossover (Sexual Reproduction)
Variants

• Most Higher Animals and Plants
• Genetic Algorithms
• Genetic Programming

27
Representation Linear, Discrete
Variants

• Usual in Genetic Algorithms
• All known organisms

28
Representation Continuous
Variants

• Usual in Evolution Strategies, and today common
  in Evolutionary Programming

• Crossover
• X,Y ? rX (1-r)Y
• r drawn from uniform distribution
• 0,1
• or -0.5, 1.5

29
Representation Tree Structured
Variants

• Genetic Programming

30
Diversity Resource Competition
Variants

• Resource competition encourages animals to
  specialise in niches

• Simulated in artificial systems
• Fitness sharing
• Very effective in encouraging diversity

31
Diversity Speciation
Variants

• In natural systems, separate species arise
  rapidly
• Separate species do not inter-breed
• Allows each gene pool to develop separately
• Herring Gulls and Black-backed Gulls do not breed

• Simulated in artificial systems
• Tag-template matching
• Not very effective in encouraging diversity

32
Diversity Spatial Distribution
Variants

• Spatial separation allows groups of animals to
  evolve separately
• A natural process leading to speciation and
  diversity
• Herring / Black-backed gulls are an example
• Two separate species in Britain, but gradual
  change around the North Pole

33
Co-evolution
Variants

• Models reality
• Useful for competitive games, social modelling
• Can encourage diversity generality

34
Where are we?UsingEvolutionarySystems
35
Why not use Evolutionary Algorithms?
Application

• Evolutionary algorithms are computationally
  expensive
• If there is already a good method, evolutionary
  methods may be unsuitable
• Use simplex for a linear programming problem
• But many problems dont have good, simple
  techniques
• We often tailor the problem to the technique
• No longer necessary
• Sometimes the cost of learning to use the
  technique is greater than the computational cost
• Evolutionary techniques are relatively easy to
  master
• Evolutionary algorithms are naturally parallel

36
Optimisation
Application

• Evolutionary methods are highly flexible

• Problems can be

• High Dimensional
• Mixed
• Deceptive

• Objective can be

37
Optimisation
Application

• Constraints can be

• Optima can be

38
Multi-Objective Optimisation
Application

• Multi-objective
• Requires multiple evaluation
• EC computational cost comparable

39
Learning Relationships
Application

• Example learning the spatial relationships
  determining animal distributions

40
Learning Games
Application

• Co-evolutionary systems
• Recreational Games
• Checkers
• Fogels Blondie
• Backgammon

41
Planning
Application

• Aler et al
• EVOCK system
• Evolves plans

42
Modelling
Application

• Evolving Systems of Equations
• Difference Equations
• Ordinary Differential Equations
• Partial Differential Equations

43
Design
Application

• Radar Beam

44
Evolvable Hardware
Application

• Adaptive Cell Phones

45
Other Population Based Algorithms

• Evolutionary algorithms are not the only
  population-based stochastic search algorithms
• Estimation of Distribution Algorithms (EDAs)
• More directly based on probability theory
• Will look at this in more detail
• Ant algorithms
• Based on the search behaviours of social insects
• ants, termites
• Similar to EDAs
• Swarm algorithms
• Based on the swarming behaviours of flocks
• insects, birds etc

46
EDA Algorithm
EDA

• Initialise the EDA probability
  modelRepeat Generate population from
  probability model Evaluate population
  fitness Generate a new probability model
  (optional) Update the probabilities in the
  model based on population fitnessUntil
  stopping criteria are satisfied

47
EDA Variants
EDA

• Simplest version (PBIL, Baluja 1994)
• Probability model is based on GA chromosome
• Each gene contains a probability table for each
  possible value
• Remember our x2 problem from last lecture
• Initial model is prior (usually uniform)
• Random sample might be 01101, 11000, 01000, 10011

48
PBIL Algorithm
EDA

• Initial population 01101, 11000, 01000, 10011
• Use truncation selection, say 50
• Selection rate is a parameter of the system
• Selected population is 11000, 10011
• Probability model of population is

49
PBIL Algorithm
EDA

• Combine prior and new probability tables
• Discount rate ?0.2 is a parameter of the system
• Note that gene values can be lost from the search
  
• If the probability goes to zero
• Setting the population (sampling) size and
  discount rate correctly is important

50
PBIL Algorithm
EDA

• Repeat the process

51
PBIL Algorithm
EDA

• Until termination condition is reached
• Note that, in this particular problem, selection
  pressure on low order bits is very weak until
  high order bits are stabilised

52
PBIL Assumptions
EDA

• PBIL makes a basic (and generally unjustified)
  assumption
• That good values for genes are independent of
  each other
• Not actually true even for this simple problem
• Very untrue for most interesting problems
• If genes are independent, its better to just
  search each gene independently
• Maybe evolutionary search is unnecessary

53
Other EDAs
EDA

• No serious EDAs now use independent gene
  assumptions
• Simplest form pairwise dependency
• Most sophisticated algorithms learn dependency
  relationships
• Learning Bayesian Networks is commonest
• Issues
• How to decide when to update the probabilty
  structure (e.g. BN)
• How to update the probability structure
• What initial probabilities to use for a new
  structure
• What Prior to use on a new structure

54
Summary

• Evolutionary Algorithms as stochastic search
  algorithms
• Biological Inspirations of Evolutionary
  Algorithms
• Variants of Evolutionary Algorithms
• Applications of Evolutionary Algorithms
• Estimation of Distribution Algorithms

55
?????