An overview of the parameterless genetic algorithm

1
An overview of the parameter-less genetic
algorithm

• Fernando Lobo
• Universidade do Algarve, Portugal
• Joint work with
• Georges Harik and David Goldberg
• IlliGAL, UIUC

2
Talk overview

• Motivation
• Review parameter setting in GAs
• Parameter-less approach
• Computer experiments

3
Motivation

• Traditional GAs are hard to use
• User must specify a number of parameters
• We should make life easier for users

4
Which parameters?

• Population size
• Selection pressure
• Crossover probability
• Mutation probability

5
Other design choices

• Representation
• Variation operators

6
Parameter setting in GAs

• Empirical studies
• Parameter adaptation techniques
• Facetwise theoretical studies

7
Empirical studies

• De Jong (1975), Schaffer et al. (1989)
• population size 50-100
• prob. crossover 0.6-0.9
• prob. mutation 0.001-0.01

8
Parameter adaptation techniques

• Parameter values change throughout the search
• Lots of work on operator probabilities
• Very little on population sizing

9
Facetwise theoretical studies

• Selection alone
• Mutation alone
• Population sizing ignoring mutation and assuming
  perfect mixing

10
Two critical facetwise studies

• Control maps
• Population sizing

11
Selection rate and crossover prob.
Goldberg, Deb, Thierens (1993)
12
Selection rate and crossover prob.

• Avoid very high and very low selection rates
• Ensure BB growth s (1-pc) gt 1
• s4, pc0.5 gt s (1-pc) 2

13

• Selection pressure
• Crossover probability
• Population size
• Mutation probability

14
Population sizing theory

• Goldberg, Deb, Clark (1991) Harik,
  Cantú-Paz, Goldberg, Miller (1997)
• Not easy to apply for real world problems
• But very important to understand the role of the
  population in GAs

15
The intuition for population sizing in GAs

• Difficult problems require more processing power
  than easy problems.
• More processing power gt larger population

16
What happens in practice?

• User guesses a population size (N), and let it
  run a number of generations (G)

17
Guessing right is luck

• What if N is too small?
• What if N is too large?

18
This is the result
19
Parameter-less GA approach

• Start with a small population size and let it run
• After some time, spawn a new population twice as
  large, and let it run
• And so on

20
Parameter-less GA approach

• Establish a race among populations of different
  sizes
• giving a head start to the smaller ones

21
16
22
16
23
16
24
16
Larger population sizes
32
25
16
Larger population sizes
32
26
16
Larger population sizes
32
27
16
Larger population sizes
32
28
16
Larger population sizes
32
29
16
Larger population sizes
32
30
16
Larger population sizes
32
31
16
Larger population sizes
32
32
4
6
5
7
9
8
3
2
16
Larger population sizes
32
33
4
6
5
7
9
8
10
3
2
16
Larger population sizes
32
34
4
6
5
7
9
8
10
11
3
2
16
Larger population sizes
32
35
4
6
5
7
9
8
10
12
11
3
2
16
Larger population sizes
32
36
4
6
5
7
9
8
10
12
11
3
2
16
Larger population sizes
32
64
37
4
6
5
7
9
8
10
12
11
13
3
2
16
Larger population sizes
32
64
38
4
6
5
7
9
8
10
12
11
13
14
3
2
16
Larger population sizes
32
64
39
4
6
5
7
9
8
10
12
11
13
15
14
3
2
16
Larger population sizes
32
64
40
4
6
5
7
9
8
10
12
11
13
15
14
3
2
16
Larger population sizes
32
64
41
4
6
5
7
9
8
10
12
11
13
15
14
16
3
2
16
Larger population sizes
32
64
42
Implementation

• Use a counter base 4
• Least significant digit changes 4 times more
  often than the next digit.

43
Do we keep running all populations forever?

• Answer No
• Sometimes populations are deleted
• Well see how in a moment

44
After some time
45
generation number
30
4
6
5
7
9
8
10
11
3
2
256

Larger population sizes
512
avg fit 12.6
1024
avg fit 11.8
avg fit 7.8
46
generation number
30
4
6
5
7
9
8
10
11
3
2
256

Larger population sizes
512
avg fit 12.6
1024
avg fit 13.2
avg fit 7.8
47
Need a tall guy(to play basketball)
12 year old
6 year old
48

• Forget about the 12 year old kid
• Hes not growing anywhere

49
generation number
30
4
6
5
7
9
8
10
11
3
2
256

Larger population sizes
512
avg fit 12.6
1024
avg fit 13.2
avg fit 7.8
50
generation number
4
6
5
7
1
3
2
Larger population sizes
Delete population of size 256, and keep
going with the others.
51
When to delete populations?

• At convergence
• When a population is overtaken fitness-wise by a
  larger population

52
Experiment 1 onemax
53
Experiment 2 noisy onemax
54
Experiment 3 trap functions
55
Application to a network expansion problem

• Goal illustrate the techniques in non-artificial
  problems
• Case study a simplified version of a utility
  network expansion problem

56
Network expansion problem
A 10-bit problem
57
The encoding
Solution 0110000110
58
Obj. function in 3 steps Step 1
59
Step 2
Construct a minimum spanning tree
60
Step 3
Network corresponding to solution 0110000110
61
60-bit network problem
Takes on average 100-200 thousand function eval.
to reach the target
62
Best solution found
63
60-bit network problemstandard settings

• population size 100, pc0.7, pm1/l
• couldnt reach the target solution after a
  million function evaluations

64
60-bit network problem

• Its unlikely that a user would guess 2000 as the
  right population sizing for this problem
• Not even a GA expert would guess it right
• Use parameter-less GA

65
A better name

• Parameter-less crossover based GA
• So far, mutation has been ignored
• But nothing prevents its use

66
Another good thing about it

• Independent of the GA to be used with
• Can have parameter-less versions of SGA, LLGA,
  ECGA, BOA, and so on

67
Extensions

• Integrate mutation
• Currently working on it
• Hope to publish results soon

68
Summary

• Motivation
• Flaws of current GA practices
• Mechanics and rationale of parameter-less GA

69
Conclusions

• It is possible to eliminate the parameters of the
  GA
• And keep a good performance across a broad range
  of problems

70
Want to know more?

• Harik Lobo (GECCO 1999 Conference)
• Lobo (PhD thesis 2000, available on web)
• Lobo Goldberg (Information Sciences Journal, in
  Press)