G5BAIM Artificial Intelligence Methods

1
G5BAIMArtificial Intelligence Methods

• Graham Kendall

Evolutionary Algorithms
2
Evolutionary Algorithms

• How can programs learn?
• This lecture is only an introduction
• This techniques can be used as an optimisation
  strategy but we are going to look at a learning
  example

3
Evolutionary Algorithms

• Chapter 8 of
• Michalewicz, Z. (1996). Genetic Algorithms Data
  Structures Evolution Programs, Springer-Verlag,
  ISBN 3-540-60676-9
• Almost anything by David Fogel
• Fogel, D. B. (1994) IEEE Transactions on Neural
  Networks, Vol 51, pp3-14
• Fogel, D.B. (1998) Evolutionary Computation The
  Fossil Record, IEEE Press, ISBN 0-7803-3481-7, pp
  3-14
• Michalewicz, Z. and Fogel, D. (2000). How to
  Solve It Modern Heuristics. Springer-Verlag,
  ISBN 3-540-66061-5

4
Evolutionary Algorithms

• There are other ways that we could design
  computer programs so that they learn
• For example, knowledge based on some suitable
  logic symbolism
• Use inference rules
• We should also be careful not confuse
  evolutionary strategies with evolutionary
  programming (EP). EP is about writing programs
  that write programs

5
Evolutionary Algorithms vs GAs

• ESs are an algorithm that only uses mutation and
  does not use crossover
• This is not a formal definition and there is no
  reason why we cannot incorporate crossover (as
  Michalewicz, 1996 shows)

6
Evolutionary Algorithms vs GAs

• ESs are normally applied to real numbers
  (continuous variables) rather than discrete
  values.
• Again, this is not a strict definition and work
  has been done on using ESs for discrete problems
  (Bäck, 1991) and (Herdy, 1991)

7
Evolutionary Algorithms vs GAs

• ESs are a population based approach
• Originally only a single solution was maintained
  and this was improved upon.

8
Evolutionary Algorithms vs GAs

• In summaryESs are
• Like genetic algorithms but only use mutation and
  not crossover
• They operate on real numbers
• They are a population based approach
• But we can break any, or all, of these rules if
  we wish!

9
Evolutionary Algorithms - How They Work

• An individual in an ES is represented as a pair
  of real vectors, v (x,s)
• x, represents a point in the search space and
  consists of a number of real valued variables
• The second vector, s, represents a vector of
  standard deviations

10
Evolutionary Algorithms - How They Work

• Mutation is performed by replacing x by
• xt1 xt N(0, s)
• N(0, s) is a random Gaussian number with a mean
  of zero and standard deviations of s

11
Evolutionary Algorithms - How They Work

• This mimics the evolutionary process that small
  changes occur more often than larger ones
• An algorithm (in C) that produces Gaussian
  random numbers is supplied in the handout

12
Evolutionary Algorithms - How They Work

• In the earliest ESs (where only a single
  solution was maintained), the new individual
  replaced its parent if it had a higher fitness
• In addition, these early ESs, maintained the
  same value for s throughout the duration of the
  algorithm
• It has been proven that if this vector remains
  constant throughout the run then the algorithm
  will converge to the optimal solution

13
Evolutionary Algorithms - How They Work

• Problem
• Although the global optimum can be proved to be
  found with a probability of one, it also states
  that the theorem holds for sufficiently long
  search time
• The theorem tells us nothing about how long that
  search time might be

14
Evolutionary Algorithms - How They Work

• To try and speed up convergence Rechenberg has
  proposed the 1/5 success rule. It can be stated
  as follows
• The ratio, ?, of successful mutations to all
  mutations should be 1/5. Increase the variance of
  the mutation operator if ? is greater than 1/5
  otherwise, decrease it

15
Evolutionary Algorithms - How They Work

• Motivation behind 1/5 rule
• If we are finding lots of successful moves then
  we should try larger steps in order to try and
  improve the efficiency of the search
• If we not finding many successful moves then we
  should proceed in smaller steps

16
Evolutionary Algorithms - How They Work

• The 1/5 rule is applied as follows
• if ?(k) lt 1/5 then s scd
• if ?(k) gt 1/5 then s sci
• if ?(k) 1/5 then s s

17
Evolutionary Algorithms - How They Work

• if ?(k) lt 1/5 then s scd
• if ?(k) gt 1/5 then s sci
• if ?(k) 1/5 then s s
• k dictates how many generations should elapse
  before the rule is applied
• cd and ci determine the rate of increase or
  decrease for s
• ci must be greater than one and cd must be less
  than one
• Schwefel used cd 0.82 and ci 1.22 (1/0.82)

18
Evolutionary Algorithms - How They Work

• Problem with the applying the 1/5 rule
• It made lead to premature convergence for some
  problems
• Increase the population size, which now turns
  ESs into a population based approach search
  mechanism

19
Evolutionary Algorithms - How They Work

• Increase population size
• The population size is now (obviously) gt 1.
• All members of the population have an equal
  probability of mating - compare with GA
• We could now introduce the possibility of
  crossover
• As we have more than one individual we have the
  opportunity to alter s independently for each
  member
• We have more options with regards to how we
  control the population (discussed next)

20
Evolutionary Algorithms - How They Work

• In evolutionary computation there are two
  variations as to how we create the new generation

21
Evolutionary Algorithms - How They Work

• (? ?), uses ? parents and creates ? offspring
• After mutation, there will be ? ? members in
  the population
• All these solutions compete for survival, with
  the ? best selected as parents for the next
  generation

22
Evolutionary Algorithms - How They Work

• (?, ?), works by the ? parents producing ?
  offspring (where ? gt ? )
• Only the ? compete for survival. Thus, the
  parents are completely replaced at each new
  generation
• Or, to put it another way, a single solution only
  has a life span of a single generation

23
Evolutionary Algorithms - How They Work

• The original work on evolution strategies
  (Schwefel, 1965) used a (1 1) strategy
• This took a single parent and produced a single
  offspring
• Both these solutions competed to survive to the
  next generation

24
Evolutionary Algorithms - Case Study

• Learning how to play Poker
• fold(x) exp(-eval(b) (x a))
• call(x) eval(c) exp(-eval(b)2 (x-a)2)
• raise(x) exp(eval(b) (x a 1.0))

25
Evolutionary Algorithms - Case Study

• Learning how to play Poker
• Two-dimensional hypercube of solutions
• Betting Position
• Risk Management
• N solutions of seven real values in each hyprcube
  element
• The poker playing agent evolves the variables

26
Evolutionary Algorithms - Case Study

• Simplified Blackjack
• Blackjack is a two player game comprising of a
  dealer and a player. The dealer deals two cards
  (from a normal pack of 52 cards) to the player
  and one card to himself. All cards are dealt face
  up. All cards take their face value, except Jack,
  Queen and King which count as 10 and Aces which
  can count as one or eleven
• The aim for the player is to draw as many cards
  as he/she wishes (zero if they wish) in order to
  get as close as possible to 21 without exceeding
  21

27
Evolutionary Algorithms - Your Go

• Simplified Blackjack
• How might we write an agent that learns how to
  play blackjack?
• Learn Probabilities?

28
Evolutionary Algorithms - Your Go

• Questions
• Do you think this would work?
• Should we use a single candidate for each
  probability or should we have a population
  greater than one?
• What sort of evolutionary scheme should we use
  (? ?) or (?, ?) and what values should we give
  ? and ??
• Can you come up with a better representation
  other than trying to learn probabilities?

29
Evolutionary Algorithms - Finally

• Evolutionary algorithms can be used as search
  methods as well as a learning mechanism
• It just needs saying!

30
G5BAIMArtificial Intelligence Methods

• Graham Kendall

End of Evolutionary Algorithms