Genetic Algorithms

1
Genetic Algorithms
2
Biological Evolution

• Lamarck and others
• Species transmute over time
• Darwin
• Consistent, heritable variation among individuals
  in population
• Natural selection of the fittest

3
Why GAs?

• Evolution is known to be a successful, robust
  method for adaptation within biological systems.
• GAs can search spaces of hypotheses containing
  complex interacting parts.
• GAs have been applied successfully to a variety
  of learning tasks and optimization problems
• GAs admit a natural implementation on massively
  parallel computers.

4
GAs - High Level Overview

• Begin with a collection of initial hypotheses a
  population
• Then score the hypotheses (individuals) in the
  current population using a fitness function.
• A new population is is then generated by
  probabilistically selecting the most fit
  individuals from the current population (survival
  of the fittest).
• Some of these selected individuals are carried
  forward into the next generation intact.
• Others are are used as basis for creating new
  offspring individuals by applying genetic
  operations of crossover and mutation.

5
Formally

• GA(Fitness, Fitness threshold, p, r,m)
• Initialize P ? p random hypotheses
• Evaluate for each h in P, compute Fitness(h)
• While max Fitness(h) lt Fitness threshold
• 1. Select Probabilistically select (1 - r)p
  members of P to add to Ps.
• 2. Crossover Probabilistically select rp/2
  pairs of hypotheses from P. For each pair, lth1,
  h2gt, produce two offspring by applying the
  Crossover operator. Add all offspring to Ps.
• 3. Mutate Invert a randomly selected bit in mp
  random members of Ps
• 4. Update P ? Ps
• 5. Evaluate for each h in P, compute Fitness(h)
• Return the hypothesis from P that has the
  highest fitness.

6
Individual Selection

• Roulette-Wheel Selection Choose members from a
  population in a way that is proportional to their
  fitness.
• Populations total fitness score is represented
  by a pie chart, or roulette wheel.
• Assign a slice of the wheel to each member of the
  population.
• Size of the slice is proportional to that
  individual fitness score.
• Select a member by spinning the ball and
  selecting the one at the point it stops.
• Tournament Selection Two members are chosen at
  random from a population.
• With some predefined probability p the more fit
  of these two is then selected, and with
  probability 1-p the less fit hypothesis is
  selected.
• Sometimes TS yields a more diverse population
  that RS.

7
Representing Hypotheses

• Represent
• (Outlook Overcast ? Rain) ? (Wind Strong)
• by
• Outlook Wind
• 011 10
• Represent
• IF Wind Strong THEN PlayTennis yes
• by
• Outlook Wind PlayTennis
• 111 10 10

Dont care for Outlook here.
8
Operators for Genetic Algorithms
9
GABIL DeJong et al. 1993

• Learn a set of rules
• Representation
• Each hypothesis is a set of rules
• To represent a set of rules, the bit-string
  representation of individual rules are
  concatenated
• Example
• IF a1 T AND a2 F THEN c T
• IF a2 T THEN c F
• a1 a2 c a1 a2 c
• 10 01 1 11 10 0

10
GABIL

• Fitness function
• Fitness(h) (correct(h))2
• correct(h) the percent of all training examples
  correctly classified

11
GABIL Genetic operators

• Use the standard mutation operator
• Crossover extension of the two-point crossover
  operator
• want variable length rule sets
• want only well-formed bitstring hypotheses
• Crossover with variable-length bitstrings
• 1. choose two crossover points for h1. Let d1
  (d2) be the distance to the rule boundary
  immediately to its left.
• 2. now restrict points in h2 to those that have
  the same d1 and d2 value

12
GABIL Crossover

• a1 a2 c a1 a2 c
• h1 10 01 1 11 10 0
• h2 01 11 0 10 01 0
• Suppose crossover points for h1, are after bits
  1, 8 (d11d23)
• a1 a2 c a1 a2 c
• h1 10 01 1 11 10 0
• Allowed pairs of crossover points for h2 are
  lt13gt, lt18gt, lt68gt.
• If pair lt13gt is chosen,
• a1 a2 c a1 a2 c
• h2 01 11 0 10 01 0
• the result is

13
GABIL Crossover

• a1 a2 c
• h3 11 10 0
• a1 a2 c a1 a2 c a1 a2 c
• h4 00 01 1 11 11 0 10 01 0