Genetic Programming: A Parallel Approach

1
Genetic ProgrammingA Parallel Approach

• Wolfgang Golubski
• University of Siegen, Germany
• Dept. of Electricial Engineering Computer
  Science

2
Structure

• 1. Motivation
• 2. Basic GP-Model
• 3. Parallel GP-Models
• 4. Master-Worker-GP-Model
• 5. Results
• 6. Conclusion

3
1. Motivation
4
1. Motivation

• Algorithm (EA, GA, GP) can solve many problems

• The users must have much experiences

• A lot of tests are necessary

• The development process is very time consuming

• gt Parallelization usable by everyone

5
2. Basic GP-Model
6
2. Basic GP-Model
Objects or Individuals

• Algorithminitialize populationevaluate
  fitnesswhile (no stop command) do recombine
  objects reproduce objects evaluate
  fitnessend

Population
7
2. Basic GP-Model Representation



x
x


x
x
x

x
x
X3 X2 2x
8
2. Basic GP-Model Recombination






C1
C2
C3
C4
D1
D2
D3

D4
D5
9
2. Basic GP-Model Recombination






C1
C2
C3
C4
D1
D2
D3

D4
D5
10
2. Basic GP-Model Recombination






C1
C2
C3
C4
D1
D2
D3

D4
D5
11
2. Basic GP-Model Recombination






C1
C2
C3
C4
D1
D2
D3

D4
D5






C1
C2
C3
C4
D1
D2
D3

D4
D5
12
3. Parallel GP-Models
13
3. Parallel GP-Model Global Single-Population
Master-Slave

• Algorithminitialize populationevaluate
  fitnesswhile (no stop command) do recombine
  objects reproduce objects evaluate
  fitnessend

14
3. Parallel GP-Model Global Single-Population
Master-Slave

• Algorithminitialize populationevaluate
  fitnesswhile (no stop command) do recombine
  objects reproduce objects evaluate
  fitnessend

15
3. Parallel GP-Model Global Single-Population
Master-Slave
Master

• Algorithminitialize populationevaluate
  fitnesswhile (no stop command) do recombine
  objects reproduce objects evaluate
  fitnessend

16
3. Parallel GP-Model Global Single-Population
Master-Slave
Master

• Algorithminitialize populationevaluate
  fitnesswhile (no stop command) do recombine
  objects reproduce objects evaluate
  fitnessend

Worker1
Worker4
Worker2
Worker5
Worker6
Worker3
17
3. Parallel GP-Model Global Single-Population
Master-Slave
Master

• Algorithminitialize populationevaluate
  fitnesswhile (no stop command) do recombine
  objects reproduce objects evaluate
  fitnessend

18
3. Parallel GP-Model Global Single-Population
Master-Slave
Master

• Algorithminitialize populationevaluate
  fitnesswhile (no stop command) do recombine
  objects reproduce objects evaluate
  fitnessend

Worker1
Worker4
Worker2
Worker5
Worker6
Worker3
19
3. Parallel GP-Model Single-Population
Fine-Grained
Each object resides on one node !!!

• Algorithminitialize populationevaluate
  fitnesswhile (no stop command) do recombine
  objects reproduce objects evaluate
  fitnessend

20
3. Parallel GP-Model Single-Population
Fine-Grained
Each object resides on one node !!!

• Algorithminitialize populationevaluate
  fitnesswhile (no stop command) do recombine
  objects reproduce objects evaluate
  fitnessend

gt Recombination only in the direct neighborhood
21
3. Parallel GP-Model Multiple-Population
Coarse-Grained

• Algorithminitialize populationevaluate
  fitnesswhile (no stop command) do recombine
  objects reproduce objects evaluate
  fitnessend

Node 1
Node 4
Node 2
Node 5
Node 6
Node 3
22
3. Parallel GP-Model Multiple-Population
Coarse-Grained

• Algorithminitialize populationevaluate
  fitnesswhile (no stop command) do recombine
  objects reproduce objects evaluate
  fitnessend

Node 1
Node 4
Node 2
Node 5
Node 6
Node 3
gt Recombination only in the neighborhood
23
4. Master-Worker GP-Model
24
4. Master-Worker GP-Model
Master
Worker 1
Worker 2
Worker 3
Worker 4
25
4. Master-Worker GP-Model
Master
Distribute GP-Param.
Worker 1
Worker 2
Worker 3
Worker 4
26
4. Master-Worker GP-Model
Master
Apply Basic GP
Worker 1
Worker 2
Worker 3
Worker 4
27
4. Master-Worker GP-Model
Master
and interrupts after a fixed number of steps
Worker 1
Worker 2
Worker 3
Worker 4
28
4. Master-Worker GP-Model
Master
Send fittest individuals
Worker 1
Worker 2
Worker 3
Worker 4
29
4. Master-Worker GP-Model
Master
Collecting them to a new fittest set
Worker 1
Worker 2
Worker 3
Worker 4
30
4. Master-Worker GP-Model
Master
Collecting them to a new fittest set
Worker 1
Worker 2
Worker 3
Worker 4
31
4. Master-Worker GP-Model
Master
distribute fittest individuals
Worker 1
Worker 2
Worker 3
Worker 4
32
5. Results
33
5. Results Symbolic Regression

• 42 functions of the form f(x) ?i1..10 ai
  xiwhere ai?Z.
• 50 sample points of -1,1 for each function
• 100 test runs for each function
• Two test series

34
5. Results Symbolic Regression
35
5. Results Symbolic Regression - Test Serie 1
36
5. Results Symbolic Regression - Test Serie 2
37
6. Conclusion
38
6. Conclusion Symbolic Regression
39
6. Conclusion

• Simple Parallel Master-Worker GP-Model
  (implemented in Java and Java RMI)
• showing promising results
• more tests on more complcx problems
• vary the synchronization steps
• the existing implementation allows in an easy way
  to vary the distribution policy of the master
  process.