Genetic Algorithms for Real Parameter Optimization

1
Genetic Algorithms for Real Parameter Optimization

• Written by Alden H. Wright
• Department of Computer Science
• University of Montana
• Presented by Tony Morelli
• 11/01/2004

2
Background

• Usual method of applying GAs to real-parameter is
  to encode each parameter using binary coding or
  Gray coding
• Parameters are concatenated together to create a
  chromosome.
• Each Bit position corresponds to a gene
• Each Bit value corresponds to an allele
• This paper's approach
• Chromosome Vector of real parameters
• Gene A real number
• Allele A real value

3
Binary Coding

• Binary Coding
• Break valid range into segments and associate
  value based on segment
• If 0 lt x lt 4 and 5 bits are used
• 32 Segments
• Each segment 1/8 (.125)
• 3/8 00011 and 7/16 00011

4
Gray Coding

• Gray Encoding
• Increase of 1 step changes only 1 bit.
• Example
• 0 0000
• 1 0001
• 2 0011
• 3 0010
• 4 0110
• Convert Bin-Gray Gray-Bin

5
Crossover

• One Point Binary Crossover
• 1 Crossover point is selected
• Bits from 1 parent and to the left of the
  crossover point are combined with bits from the
  other parent and to the right of the crossover
  point
• Crossing over 5/32 (00101) and 27/32 (11011)
  between bits 3 and 4 yields 7/32 (00111) and
  25/32 (11001)

6
Mutation

• Binary Coded GA
• Probability of mutation is low
• If mutation occurs, bit changes from 1 to 0 or 0
  to 1
• If change from 0 to 1
• Binary coding Change is in the positive
  direction
• Gray coding Change in either direction
• If change from 1 to 0
• Binary coding Change is in the negative
  direction
• Gray coding Change is in either direction

7
Schemata

• Similarity Template
• Describes a subset of the space of chromosomes
• 01 010, 011
• Connected schemata are the most meaningful
• They capture locality info about the function

8
A Real Coded Genetic Algorithm

• Standard GA
• 1. A method for choosing the initial population
• 2. A Scaling function that creates a nonnegative
  fitness function
• 3. Find the sampling rate of an individual
• 4. Pick which individuals are allowed to
  reproduce
• 5. Reproduction operators to produce new
  individuals
• 6. A method for choosing which reproduction
  operator to apply

9
A Real Coded Genetic Algorithm

• Standard GA, only steps 5 and 6 require bitwise
  manipulation.
• Real crossover is almost the same is in binary
• Take the list of real numbers from one parent,
  combine them with a list from the other parent
• 5,6,7,8,1,2,3,4 combine at crossover point
  between 2 and 3 to create children 5,6,3,4 and
  1,2,7,8

10
A Real Coded Genetic Algorithm

• Real Mutation
• Mutation is performed if chromosome is selected
• Direction is then chosen (50/50 either positive
  or negative
• Amount of mutation is determined
• Original parameter is x, range a,b, mutation
  size M
• Direction is positive
• Mutated parameter is uniformly chosen from
  x,min(M,b)

11
A Real Coded Genetic Algorithm

• Problems with real crossover

12
A Real Coded Genetic Algorithm

• Linear Crossover
• From 2 parent points, 3 new points are generated
• (1/2)p1 (1/2)p2, (3/2)p1 - (1/2)p2,
  (-1/2)p1(3/2)p2
• (1/2)p1 (1/2)p2 is the midpoint of p1 and p2
• The others are on the line determined by p1 and
  p2
• The best 2 of the 3 points are sent to the next
  generation
• Disadvantage - Highly disrupted of schemata and
  is not compatible with the schema theorem
  described in the next slide.

13
Schemata Analysis for Real-Allele Genetic
Algorithms

• Restrict some or all of the parameters to
  subintervals of their possible ranges
• Ii denotes the interval
• If parameter space is -1,1x-1,1x-1,1 and
  schema is -1,1x0,1x-1,0, the m-tuple would
  be I2I3
• The probability of an individual being selected
  for reproduction is the ration of its fitness to
  the average fitness of the entire population

14
Schemata Analysis for Real-Allele Genetic
Algorithms

• The expected proportion of individuals of schema
  s that are selected after reproduction is shown
  by

15
Experimental Results

• Tested on DeJongs 5 problems, 2 other problems
  (Schaffer, Caruana, Eshelman, and Das)

16
Experimental Results

• 2 Point Crossover
• The Elitist Strategy
• Bakers Selection Procedure
• Population size of 20
• Crossover rate of 0.8
• Gray coding was used for Binary-Coded
• GA was run for 1000 trials, except for one of the
  cases which was run for 5000
• Best Performance Min value over all trials
• Best Offline Performance Average over function
  evaluations of the best value obtained up to that
  function evaluation

17
Experimental Results

• Multiple runs were done to tune parameters
• Binary
• 1000 experiments at each mutation rate from 0.005
  to 0.05 in steps of 0.005
• Real
• 1000 experiments were done at each combination of
  a mutation size and mutation rate
• Mutation sizes 0.1-0.3 steps of 0.1
• Mutation rate from 0.05 to 0.3 steps of 0.05
• 50 real crossover and 50 linear crossover

18
Experimental Results
19
Experimental ResultsSummary

• Real Coded algorithm with 50 real crossover and
  50 linear crossover performed better than 100
  real crossover on all problems
• Real-Coded with both types of crossover performed
  better than the binary-coded on 7/9 problems
• Real-Coded algorithm with real crossover
  performed better than the binary-coded on 5/9
  problems.

20
Experimental ResultsSummary

• On problems F4 and F7 the mixed crossover
  real-coded did much better than the other 2
• On F5 (Shekel's Foxholes) the binary coded
  algorithm did much better than the real-coded
  algorithms.
• This problem is well suited for binary GAs
• When it was rotated 30 degrees, the difference
  between the real-coded and the binary was less,
  but the binary GA still outperformed.

21
Conclusions

• Results showed that the real-coded GA based on a
  mixture of real and linear crossover gave
  superior results to binary-coded GAs on most test
  problems
• Real-Coded GA with both linear and real crossover
  outperformed the GA with only 1 of them.
• Strengths of a real-coded GA
• 1. Increased Effeciency (No need to convert bit
  strings)
• 2. Increased Precision (Using real numbers)
• 3. Can use different mutation and crossover
  techniques

22
Questions/Comments

• Word of the day ENVISAGE
• envisage
• 1. To conceive an image or a picture of,
  especially as a future possibility envisaged a
  world at peace.
• 2. To consider or regard in a certain way.