A Gentle Introduction to Evolutionary Computation

1
A Gentle Introduction to Evolutionary Computation

• Keith L. Downing
• Department of Computer Science
• The Norwegian University of Science Tech
• Trondheim, Norway

2
Outline

• Generic Evolutionary Algorithm
• Types of EAs
• GA examples
• GP examples
• EAs Machine Learning (ML)
• EAs Intelligent Agents

3
Darwinian Evolution
Physiological, Behavioral
Phenotypes
Natural Selection
Ptypes
Reproduction
Gtypes
Genotypes
Genetic
4
Evolutionary Algorithms
Semantic
Parameters, Code, Neural Nets, Rules
Performance Test
P,C,N,R
Generate
Bits
Bit Strings
Syntactic
5
Evolutionary Computation Parallel Stochastic
Search
Indiv 1 2 3 4 5 6
Biased Roulette Wheel
Fitness 3 8 2 4 1 1
6
1
5
Selection Biasing
4
2
Translation Performance Test
3
Selection
Crossover
Mutation
6
Types of Evolutionary Algorithms

• Genetic Algorithms (Holland, 1975)
• Representation Bit Strings gt Integer or real
  feature vectors
• Syntactic crossover (main) mutation
  (secondary)
• Evolutionary Strategies (Recehenberg, 1972
  Schwefel, 1995)
• Representation Real-valued feature vectors
• Semantic mutation (main) crossover (secondary)
• Evolutionary Programs (Fogel, Owens Walsh,
  1966 Fogel, 1995)
• Representation Real-valued feature vectors or
  Finite State Machines
• Semantic mutation (only)
• View each individual as a whole species, hence
  no crossover
• Genetic Programs (Koza, 1992)
• Representation Computer programs (typically in
  LISP)
• Syntactic crossover (main) mutation (secondary)

7
Search Methods (Michalewicz Fogel, 2000)
ML uses them all.
Traditional(Serial)
Evolutionary (Parallel)
Traditional methods can be parallel, but its
usually an independent parallelism
Whole Solutions
Partial Solutions
Whole Solutions
GA GP ES EP
Local
Exhaustive
Linear Proging
8
Using Evolutionary Algorithms

• When
• Large, rough search spaces
• Satisficing, optimization or general design
  problems
• Entire solutions are easily generated and tested
• Exhaustive search methods are too slow
• Heuristic search methods cannot find good
  solutions (e.g. Stuck at local max)
• How
• Determine representation of solutions that
  tolerates mutation and crossover.
• Define fitness function that gives graded
  evaluations - not just good/bad rating.
• Define selection function roulette-wheel
  biasing function (f fitness -gt area)
• Set key EA parameters population size, mutation
  rate, crossover rate, generations, etc.
• EAs are easy to write, and theres lots of
  freeware!
• Specific problems often require specific
  representations genetic operators

9
The Fitness Landscape of the Search Space

• Genotype representation determines the size and
  density of the search space
• Fitness function determines its texture (rough,
  smooth, etc.).
• Rough landscapes are harder to search, since
  the partial information provided by
• the fitness function does not have good
  heuristic value. E.g. A local max gets a
• high fitness score though it may be quite
  distant from the global max.

Fitness
Fitness Func
(Testing is performed at this level)
Phenotype
Genotype (Search is performed at this level)
10
Classic Genetic Algorithm

• Each chromosome may represent
• Parameters for a controller,
• Room s for an exam scheduler
• Weights for an artificial neural network
• Instruction codes for a computer program

11
Process Scheduling (Kidwell, 1993)

• Task pairs (run communicate result) to be run
  on a set of processors
• ((7, 16) (11, 22) (12, 40), (15, 22).)
• A tasks run must finish before result sending
  begins
• All processors share a central communication line
  (bus)
• Each processor can handle only one task at a
  time.
• Each processor is capable of running any of the
  tasks
• Only one processor at a time can send its message
• A task cannot be removed from a processor until
  both run send are finished
• Tasks run on the main processor, P0, require no
  communication time, whereas tasks run on all
  other processors must send their message to P0
• Goal Schedule the tasks on processors so as to
  minimize the total timespan

12
Process Schedule Optimization using the Genetic
Algorithm

• Use GA to search the space of possible schedules
  (solutions)
• 1. Represent schedule in a GA-amenable form (i.e.
  linearize it)

2. Define a fitness function Fitness
MaxTimespan - Timespan Lower timespan gt
Higher fitness Other possibilities 1/(1
(Timespan - MinTimespan))
13
GA-based Schedule Optimization

• Compute schedules timespan by running on a
  process-network simulator.
• 1. Remove next task (whose assigned
  processor is open) from task-list and
• start simulating it on that
  processor.
• 2. Remove tasks from processors as
  soon as they finish running sending
• 3. Define a biasing function to convert fitness
  to a proportion of the roulette wheel
• Sigma Scaling (One of many standard biasing
  functions)
• ExpVal(x) Max ( 0, 1 (Fitness (x) -
  AvgFitness) / (2 StDevFitness))
• Normalizing
• Roulette-Wheel(x) ExpVal(x) /
  SumofAllExpVals
• 4. Select Mutation and Crossover Rates pmut
  0 .01 pcross 0.75
• 5. Select a population size popsize 10
• 6. Select of generations numgen 10
• 7. Run the Genetic Algorithm
• Generates a random initial population (of
  schedules) and evolves them via sigma-scaling
  selection, crossover and mutation for numgen
  generations

14
Kidwells (1993) Task List

• ((7 16) (11 22) (12 40) (15 22) (17 23)
• (17 23) (19 23) (20 28) (20 27) (26 27)
• (28 31) (36 37) (31 29) (28 22) (23 19)
• (22 18) (22 17) (29 16) (27 16) (35 15))
• MaxTime Sum of all run send times 916 time
  units
• Use 8 processors P0 - P7, with P0 being the
  master processor

15
Population of Schedules (Generation 0)

• These are randomly-generated
• Processor List
• 1 Span 401 Fitn 515 (11.0) (1 2 1 3 6 7 2 0
  0 4 5 1 6 0 5 5 4 7 4 6)
• 2 Span 383 Fitn 533 (13.7) (1 0 1 0 0 4 0 1
  0 5 3 6 2 5 5 5 1 3 5 7)
• 3 Span 426 Fitn 490 ( 7.1) (6 2 2 7 6 5 2 6
  7 6 6 5 3 4 0 1 0 2 0 7)
• 4 Span 415 Fitn 501 ( 8.8) (5 0 5 7 1 2 5 4
  2 6 3 6 2 0 0 4 1 7 3 1)
• 5 Span 377 Fitn 539 (14.7) (4 2 0 6 2 1 2 0
  1 6 3 2 2 3 3 4 0 3 0 1)
• 6 Span 435 Fitn 481 ( 5.7) (3 2 7 7 6 0 3 1
  4 7 7 5 1 0 4 6 5 5 5 6)
• 7 Span 439 Fitn 477 ( 5.1) (0 3 2 3 2 2 7 0
  4 5 2 1 6 3 1 7 1 0 3 2)
• 8 Span 410 Fitn 506 ( 9.6) (2 5 2 1 0 0 2 2
  5 0 1 3 6 1 3 3 4 6 2 0)
• 9 Span 337 Fitn 579 (20.9) (1 0 0 7 0 4 3 1
  1 0 0 2 4 1 2 4 6 4 7 6)
• 10 Span 449 Fitn 467 ( 3.5) (2 2 2 4 1 4 4 6
  6 4 7 4 0 6 2 5 2 7 7 7)
• Avg Fitness 508.80 StDev Fitness 32.28

16
Roulette Wheel (Generation 0)
17
Population of Schedules (Generation 1)

• 1 Span 366 Fitn 550 (12.1) (1 0 0 3 0 4 2 1
  0 1 3 6 4 5 7 4 5 7 5 6)
• 2 Span 406 Fitn 510 ( 7.9) (1 0 1 4 0 4 1 1
  1 4 0 2 2 1 0 5 2 0 7 7)
• 3 Span 377 Fitn 539 (11.0) (4 2 0 6 2 1 2 0
  1 6 3 2 2 3 3 4 0 3 0 1)
• 4 Span 464 Fitn 452 ( 1.8) (1 0 1 3 0 4 2 0
  1 4 4 2 4 1 5 5 4 6 4 6)
• 5 Span 366 Fitn 550 (12.1) (1 2 0 7 6 7 3 1
  0 0 1 1 6 0 2 4 6 5 7 6)
• 6 Span 337 Fitn 579 (15.2) (1 0 0 7 0 4 3 1
  1 0 0 2 4 1 2 4 6 4 7 6)
• 7 Span 337 Fitn 579 (15.2) (1 0 0 7 0 4 3 1
  1 0 0 2 4 1 2 4 6 4 7 6)
• 8 Span 415 Fitn 501 ( 7.0) (5 0 5 7 1 2 5 4
  2 6 3 6 2 0 0 4 1 7 3 1)
• 9 Span 465 Fitn 451 ( 1.7) (5 5 7 7 1 2 5 2
  3 6 1 6 2 1 0 5 1 7 2 1)
• 10 Span 329 Fitn 587 (16.0) (2 0 0 1 0 0 2 4
  4 0 3 3 6 0 3 2 4 6 3 0)
• Avg Fitness 529.80 StDev Fitness 47.43

18
Roulette Wheel (Generation 1)
19
Population of Schedules (Generation 2)

• 1 Span 337 Fitn 579 (11.7) (1 0 0 7 0 0 6 0
  2 6 3 6 0 4 1 4 1 7 1 1)
• 2 Span 465 Fitn 451 ( 0.0) (5 0 5 3 1 6 1 5
  0 1 3 6 6 1 6 4 5 7 7 6)
• 3 Span 366 Fitn 550 ( 8.8) (1 2 0 7 6 7 3 1
  0 0 1 1 6 0 2 4 6 5 7 6)
• 4 Span 329 Fitn 587 (12.5) (2 0 0 1 0 0 2 4
  4 0 3 3 6 0 3 2 4 6 3 0)
• 5 Span 329 Fitn 587 (12.5) (2 0 0 1 0 0 2 4
  4 0 3 3 6 0 3 2 4 6 3 0)
• 6 Span 270 Fitn 646 (18.4) (1 0 0 5 0 0 2 0
  0 0 3 3 4 0 2 2 4 6 3 2)
• 7 Span 364 Fitn 552 ( 9.0) (2 0 0 3 0 4 3 5
  5 0 0 2 6 1 3 4 6 4 7 4)
• 8 Span 337 Fitn 579 (11.7) (1 0 0 7 0 4 3 1
  1 0 0 2 4 1 2 4 6 4 7 6)
• 9 Span 347 Fitn 569 (10.7) (5 0 1 7 1 6 1 4
  0 0 1 2 2 0 0 4 5 7 7 0)
• 10 Span 410 Fitn 506 ( 4.5) (1 0 4 7 0 0 7 1
  3 6 2 6 4 1 2 4 2 4 3 7)
• Avg Fitness 560.60 StDev Fitness 49.61

20
Roulette Wheel (Generation 2)
21
Population of Schedules (Generation 6)

• 1 Span 263 Fitn 653 (10.6) (1 0 0 7 0 4 6 0
  0 4 3 7 0 0 0 0 5 6 3 6)
• 2 Span 232 Fitn 684 (16.7) (1 0 0 5 0 0 2 0
  0 6 3 3 0 0 2 2 0 7 1 0)
• 3 Span 325 Fitn 591 ( 0.0) (1 0 0 5 0 0 6 0
  0 2 3 3 4 0 3 4 5 7 3 2)
• 4 Span 261 Fitn 655 (11.0) (1 0 0 7 0 0 2 0
  0 0 3 2 0 0 2 0 1 6 3 2)
• 5 Span 249 Fitn 667 (13.3) (1 0 0 5 0 0 2 0
  0 2 3 6 0 0 3 0 5 6 3 2)
• 6 Span 255 Fitn 661 (12.1) (1 0 0 5 0 0 6 0
  0 0 3 3 0 0 1 0 5 7 3 2)
• 7 Span 292 Fitn 624 ( 4.8) (1 0 0 7 0 0 3 0
  0 0 3 3 4 0 0 4 1 6 3 2)
• 8 Span 269 Fitn 647 ( 9.4) (1 0 0 7 0 4 6 0
  0 4 3 7 0 0 2 0 4 6 3 6)
• 9 Span 247 Fitn 669 (13.7) (1 0 0 5 0 0 2 0
  0 0 3 7 4 0 0 4 1 6 3 2)
• 10 Span 274 Fitn 642 ( 8.4) (1 0 0 7 0 0 2 0
  0 0 3 3 0 0 3 0 5 6 3 2)
• Avg Fitness 649.30 StDev Fitness 24.87

22
Roulette Wheel (Generation 6)
23
Population of Schedules (Generation 9)

• 1 Span 265 Fitn 651 ( 0.4) (1 0 0 5 0 0 2 0
  0 2 3 6 0 0 1 0 5 7 3 2)
• 2 Span 241 Fitn 675 (12.3) (1 0 0 7 0 0 2 0
  0 4 3 3 0 0 3 0 0 6 1 0)
• 3 Span 238 Fitn 678 (13.8) (1 0 0 5 0 0 6 0
  0 6 3 3 0 0 1 0 1 6 1 0)
• 4 Span 258 Fitn 658 ( 3.8) (1 0 0 5 0 0 6 0
  0 6 3 2 0 0 0 0 0 7 1 0)
• 5 Span 248 Fitn 668 ( 8.8) (1 0 0 5 0 0 2 0
  0 2 3 3 0 0 1 0 1 6 3 0)
• 6 Span 246 Fitn 670 ( 9.8) (1 0 0 5 0 0 6 0
  0 6 3 7 0 0 1 0 5 7 1 2)
• 7 Span 246 Fitn 670 ( 9.8) (1 0 0 5 0 0 2 0
  0 4 3 3 0 0 1 0 5 7 1 2)
• 8 Span 242 Fitn 674 (11.8) (1 0 0 5 0 0 6 0
  0 2 3 6 0 0 3 0 1 6 3 0)
• 9 Span 226 Fitn 690 (19.7) (1 0 0 5 0 0 2 0
  0 0 3 6 0 0 3 0 5 6 3 2)
• 10 Span 246 Fitn 670 ( 9.8) (5 0 0 5 0 0 2 0
  0 2 3 6 0 0 3 0 5 7 3 2)
• Avg Fitness 670.40 StDev Fitness 10.06
  (Convergence)

24
Roulette Wheel (Generation 9)
25
Process-Schedule Evolution
GA falls off a peak
Convergence
26
Tantrix-GA

• Downing (2005)

27
Sudoku-GA
28
Permutation Bases
Puzzle
1 2 3 6 7 8 9
1 4 6 8 9
1 2 3 5 7 9
2 4 5 6 9
1 3 4 5 6 7
3 4 5 7 8
2 4 6 7 8 9
2 4 5 7 8
1 2 3 4 6 8 9
9 3 6 2 7 8 1
6 8 4 1 9
7 5 9 2 3 1
GA Chromosome
29
GA Chromosome (Genotype)
Development
7 5 9 2 3 1
6 8 4 1 9
Phenotype
Puzzle
9 7 3 6 .. 6 5 2 2 .. 7 3 . 8
8 4 4 . 1 1 . 5 9
9 3 6 2 7 8 1
30
Mutation Simple Swapping
9 3 6 2 7 8 1
9 8 6 2 7 3 1
Perform on one or more of the permutations in a
chromosome
31
Partial-Mapping Crossover (PMX)
Mapping
Parent 1
Parent 2
9 3 6 2 7 8 1
2 3 8 9 1 6 7
3 6 2
3 8 9
2 3 8 9 7 8 1
9 3 6 2 1 8 7
White numbers need the mapping to achieve a
unique value.
Invented for TSP.
Kids
Perform on one or more corresponding permutations
in two chromosomes
32
Fitness (Part 1)

• 9 7 1 8 4 6 3 2 5
• 3 4 6 5 2 7 9 8 3
• 8 5 2 9 3 1 6 4 7
• 1 2 5 3 7 4 8 9 6
• 7 3 9 6 8 5 4 1 2
• 6 8 4 2 1 9 5 7 9
• 4 1 3 7 5 8 2 6 1
• 2 6 7 4 9 3 1 5 8
• 5 9 8 1 6 2 7 3 4

9
8
9
9
SubTotal 78
9
8
8
9
9
Count unique elements in each row
33
Fitness (Part 2)
9
9

• 9 7 1 8 4 6 3 2 5
• 3 4 6 5 2 7 9 8 3
• 8 5 2 9 3 1 6 4 7
• 1 2 5 3 7 4 8 9 6
• 7 3 9 6 8 5 4 1 2
• 6 8 4 2 1 9 5 7 9
• 4 1 3 7 5 8 2 6 1
• 2 6 7 4 9 3 1 5 8
• 5 9 8 1 6 2 7 3 4

8
9
8
9
8
9
SubTotal 78 Total 156
9
and in each 3x3 square
34
Blondie 24

• Fogel Chellapilla (2002)
• Co-evolving neural nets (the weights) to play
  checkers

Output (Move)

• Took months of co-evolution to train
• Beat Chinook (world champion - also a computer
  program)..once.
• Achieved very high rating on zone.com (continuous
  on-line checkers tournament)

35
Swarm Intelligence

• Follow Trail
• Find Food
• Make Trail

36
Evolving Populations ofSocial Insects to
PerformAnnular Sorting
Andre Hei Vik
Vegard Hartmann
P Pick up D Deposit F Forward B
Backward L Left R Right
Acting
Sensing
37
Fitness Evaluation
38
Three-object annular structure
39
Genetic Programming (GP)

• Genotype a computer program (often in Lisp)
• Phenotype Genotype (usually)

(p 2 (m (n 3 9)))
(f (h 1) (g 4 6))
Cross
over
(p 2 (h 1))
(f (m (n 3 9)) (g 4 6))
40
Symbolic Regression with GP (Koza, 1993)

• Given a set of (X,Y) pairs.
• Find a functional expression that predicts Y,
  given X.
• GP primitives
• Terminals X
• Functions , -, , , SIN, COS, EXP, RLOG
• Fitness Function
• Fitn 1/(MappingError 1)
• Sample Individual/Solution
• (- (COS ( X ( X X))) (EXP X ( ( X X ) X)))

Target
Best of Gen
Generation 0
Generation 12
Generation 50
41
Robot Wall-Following with GP (Koza, 1993)

• Given Odd-shaped room with robot in center.
• Find A control strategy for the robot that makes
  it move along the periphery.
• GP Primitives
• Terminals S0, S1..S11 (12 sensor readings,
  distance to wall),
• Functions IFLTE (if less than or equal),
  PROGN2, MF, MB (move forward/back), TL, TR (turn
  left/right).
• Fitness Function
• Fitn peripheral cells visited.
• Sample Individual/Strategy
• (IFLTE S3 S7 (MF) (PROG2 MB (IFLTE S4 S9 (TL)
  (PROG2 (MB) (TL)))))

42
Wall-Following Evolution
43
GP Analog Circuit Design (Koza et. al., 1999)

• Given Desired I/O behavior for a circuit
• Find The circuit topology and component sizes
• Approach
• Begin with an embryo circuit
• Interpret the genome recipe for growing the
  phenotype circuit from the embryo
• Use SPICE (a circuit simulator) for
  fitness-testing the phenotype

44
List
R
C
Series
5
2
End

• Execute GP tree in breadth-first manner
• Write heads link code tree to developing circuit.
• After a command executes, it passes on its write
  head to SOME of its children it produces extra
  heads if more than one kid needs it.

45
Growing a Circuit
46
21
12
47
8
21
21
50
12
12
48
(Some of) Kozas (Many) Success Stories

• Rediscovery of various famous/patented topologies
  for low-pass and high-pass filters.
• Design of amplifiers
• Design of real-time analog circuit for
  time-optimal robot control.
• Design of an electric thermometer.
• Design of a controller that optimizes with
  respect to 5 different criteria - beats classic
  textbook examples
• Discovery of protein motifs (amino-acid
  fingerprints) that are as good or better than
  those discovered by genetic engineers.
• Discovery of metabolic pathways that best explain
  time- series of chemical concentrations inside
  living cells.
• BTW Koza uses a 1000-node Boewulf!

49
Evolving Morphology Control (Karl Sims)
Morphology
Genotype
Phenotype
3d Segments Joints
Segment Graph
50
Control
Genotype Phenotype
Neural Circuit Graphs
Each segment brain have a controller
- Joint-Angle Sensors - Joint-Force Effectors
Tasks Swimming, Running, Jumping,
Capturing Search Evolution - Selection,
Mutation, Crossover
51
Evolutionary Algorithms Machine Learning

• EAs are very useful ML techniques, but they are
  quite different from other ML approaches.
• Generate-and-Test (GT)
• Conventional ML philosopy is to include as much
  intelligence in G as possible, so it only
  produces fairly good solutions (S). In contrast,
  EAs include a lot of randomness in G, and let T
  do the work (just as in nature) to filter the bad
  solutions that G may produce.

ML
G
T
Box size amount of computational effort used
EA
T
S
G
S
52
EA ML Representation

• ML methods typically involve one representation
  type, or, stated another way, ML methods are
  often representation dependent.
• Logic
• Semantic Networks
• Decision Trees
• Production rules
• Artificial Neural Networks
• EAs can use any representation type - assuming
  special rep-dependent mutation and crossover
  operators a decoder to run (reason with) the
  expressions that are generated (during fitness
  evaluation).
• Why are EAs more general?
• ML methods have a deep semantic understanding of
  the expressions being used in the sense that they
  can both reason with them and MODIFY them (i.e.
  generate new ones) in an intelligent,
  goal-directed manner.
• EAs only have a shallow syntactic understanding
  of a representation such that they can generate
  legal expressions, but often in a more random and
  less intelligent manner.

53
EA ML Solution Generators

• MLs intelligent generators
• Specializing generalizing logical concept
  descriptions by adding/dropping conjuncts and
  disjuncts.
• Adding nodes to a decision tree.
• Tuning weights or adding hidden layers to an ANN
• Intelligent in the sense that the changes are
  governed by knowledge, constraints, goals and
  previous performance. The newly-generated
  expressions thus have a reasonably good chance of
  being better than their predecessors.
• EAs dumb generators
• Crossover
• Mutation
• Inversion
• Generalizing and specializing are accidental
  side-effects of these random operations. EAs
  rely on fitness testing and selection mechanisms
  to separate the good from the bad.

54
Evolutionary Algorithms Intelligent Agents

• Most EA apps are engineering or
  operations-research search problems.
• But, low-level intelligent agents are often
  neural-net based
• Hard to train with backprop in natural
  environments.
• GAs often used to evolve parameters for these
  nets.
• However, this can be very computationally
  intensive
• Large populations thousands of generations
• Each individual tested
• In many situations to avoid domain-specific
  action and promote evolution of general
  strategies.
• Long enough for the agent to learn (if desired)
• As part of a co-evolving cooperative group (e.g.
  social insects)

55
EA IA (Continued)

• Typical EA products are excellent at some very
  specific task, but a new EA run is needed to
  handle a new situation.
• A long way from generally intelligent systems.
• Lack of constrained, intelligent design often
  leads to strange designs (ok - e.g. antenni) and
  loophole exploitation (not good, but funny!)
• Solutions often very hard to analyze explain.
• Eas very useful for designing and/or tuning
  components of an intelligent system.
• But probably not the whole system
• Unless focus is low-level intelligence, e.g.
  Artificial Life
• Many prominent EA people dislike AI.

56
Evolutionary Robotics

• Using EAs to design controllers (and occasionally
  morphologies) of autonomous robots.
• Controllers are often ANNs.
• Follows the behavior-based robotics paradigm
• Bottom up approach
• Minimal representations of the world. The world
  is its own best model (Rodney Brooks, MIT).
• Distributed, self-organizing no fixed command
  hierarchy.
• Chromosomes encode
• Weights, topologies, learning rules for ANNs
• Fitness
• Place real or simulated robot in an environment
  and give fitness points based on its ability to
  perform a task.
• Often evolve hundreds of generations in
  simulation, then download the best controllers
  into real robots and evolve a little more
  on-board.

57
Evolutionary Robotics Achievements

• Wall followers - a whole bunch!
• Foragers
• Predators Prey - often coevolving
• Miniature Airplanes.
• Robocup players.