Comparison of Pulling Back and Penalty Methods for Constraints in DPMBGA

1
Comparison of Pulling Back and Penalty Methods
for Constraints in DPMBGA

• ? Hisashi Shimosaka (Doshisha University)
• Tomoyuki Hiroyasu (Doshisha University)
• Mitsunori Miki (Doshisha University)

2
Structural Optimization Problem

• Structural optimization problem
• is a problem to design the optimum structures.
• Objective Function Volume, Cost, Weight
• Constraint Stress, Displacement, Buckling
• Features
• The landscape of the objective function has
  several local optima.
• The problems have several types of constraints.
• The problems are large-scaled due to many design
  variables.
• The feasible region is very narrow compared to
  the design field.
• Optimization algorithm
• should have an efficient searching ability for
  global search.
• should have an efficient mechanism to deal with
  the constraints.

Genetic Algorithm (GA) Mechanisms to deal with
the constraints
3
Target of our research

• To derive good solutions by GAs
• The information of the good parents should be
  inherited to the children.
• The diversity of the population should be
  maintained during the search.
• The children should be generated by consideration
  of the correlation among the design variables
• Mechanism to deal with constraints
• Penalty method
• Pulling back method
• DPMBGA to the structural optimization problems
• Penalty method and pulling back method are added
  to DPMBGA.
• The searching abilities of the both methods are
  compared through the truss structural
  optimization problems.

Distributed PMBGA (DPMBGA)
4
PMBGA
Probabilistic Model-Building Genetic Algorithm
(PMBGA)
Estimation of the distribution

1. Select promising individuals

Individual
(2) Construct a probabilistic model
Population
Probabilistic model
(3) Generate new individualsand substitute them
for old individuals

• New individuals are generated from the estimated
  probabilistic model instead of the crossover and
  mutation.

The information of the selected individuals can
be inherited to the generated individuals.
5
DPMBGA

• Distributed PMBGA (DPMBGA) Hiroyasu,2003
• Real-Coded PMBGA
• Island Model (Distributed GA)
• Probabilistic model constructed byPrincipal
  Component Analysis (PCA) and normal distributions

The diversity of the populationcan be maintained.
New individuals are generated by consideration
of the correlation among the design variables.
DPMBGA can derive better solutionsthan UNDXMGG
in some test functions.
6
Overview of the DPMBGA operations
x2
(1) Individuals with better fitness
values are selected.
Island
v1
v2
x1
(2) Individuals are transferred into the space
where there is no correlation among the
design variables using PCA.
(4) New individuals are transferred into
the original space.
(3) new individuals are generated from
normal distributed model.
7
Sampling individuals
x2
(1) Some individuals are selected
Sample individuals
Island
x1

• Sample individuals
• Individuals with the best fitness values in the
  islandare chosen as the sample individuals.

8
Individuals are Transferred into the new space
x2

• The purpose
• New individuals are generatedby consideration of
  the correlation among the design variables.
• The flow of the operation
• Principal Component Analysis (PCA) isperformed
  and vector V is obtained.
• The individuals are rotated into the space where
  there is no correlationamong the design
  variables using vector V.

v1
v2
x1
9
New individual generation

• Generation of the new individuals
• The distribution of the individuals is estimated
  using normal distributions on each design
  variable.
• The design variables of the new individuals are
  generated independently.
• Substitution for the old individuals
• New individuals are moved back to the original
  space.
• They are substituted for all of the old
  individuals.

Island
(4) New individuals are transferred into
the original space.
(3) new individuals are generated from
normal distributed model.
10
Features of DPMBGA

• Features
• Real-coded probabilistic model-building GA
• The diversity of the population is maintained by
  island model.
• New individuals are generated by consideration of
  the correlation among the design variables using
  PCA.
• The distribution of the sample individuals is
  estimated using normal distributions.
• DPMBGA is superior to the typical real-coded GA,
  UNDXMGG in the several types of test functions.
• DPMBGA is applied to structural optimization
  problems.
• DPMBGA does not have a mechanism for dealing with
  constraints.

Penalty method and pulling back method are added
to the DPMBGA
11
Penalty method

• When an individual violates the constraints,the
  objective function is added the penalty.
• The method is easy to implement.
• It is difficult to obtain the appropriate penalty
  parameter.
• When the feasible region is very narrow, it is
  difficult to search effectively.

Minimize Such that
The objective function is modified.
Minimize
12
Problem of the penalty method
Penalty method is the most popular method to deal
with constraints.
Estimation ofthe distribution
Generation of new individuals

• Some new individuals violates the constraints.
• An effective probabilistic model can not be
  constructed because the feasible region is very
  narrow near the boundary.
• The penalty method may not be performed an
  efficient search in the DPMBGA.

13
Pulling back method

• An individual that violates the constraints is
  moved to the nearest point in the feasible
  region. Mimura,2002
• The violated constraints are linearized.
• The distance to the feasible region is minimized.
• Terminal criteria of the pulling back
• All the constraints are satisfied.
• The number of the operation exceeds a certain
  number.
• The distance after the operation is smaller than
  the preset distance.

minimize Such that
If the individual still violates the constraints,
the penalty method is applied.
14
Numerical example

• The penalty method and the pulling back method
  are addedto the DPMBGA
• Comparison of the searching ability of both
  methods
• Discussion on the comparison of the results
• Truss structural optimization problem
• Design variables Areas of each
  member
• Objective function Minimization
  of the volume
• Constraints Stress is less than
  4e7 Pa. Buckling should not occur.

15
Comparison of the penalty and pulling back method

• Discussion on the affect of the number of islands
• Number of individuals per island is 16
• Number of islands is 1,2,4,8,16 and 32.
• The searching ability of the both method is
  determined by
• the number of the optimum found
• the number of the evaluations when the optimum is
  found.

Population size is changed from 16 to 512.
Parameters of the DPMBGA
Number of elites 1
Sampling Rate 0.25
Amp. of variance 2.0
Mutation rate 0.1/ (Dim. of function)
Migration interval 5
Migration Rate 0.0625
Terminal criterion(Number of evaluations) 1.0e62.0e6
Number of trials 25
Parameters of the both method
Maximum time ofpulling back 20
Minimum distanceof one pulling back 1e-8 (m)
Penalty parameter(?) 1e6
16
Number of the optimum found

• 2-Stages Truss (10 design variables)
• 25 trials
• The penalty method can derive the optimum with
  the large number of islands.
• The pulling back method can derive the optimum
  with not only the large number but also the
  small number of islands.

Islands (Population size) Penalty method Pulling back method
1 (16) 0 25
2 (32) 6 25
4 (64) 14 25
8 (128) 18 25
16 (256) 22 25
32 (512) 25 25
17
Number of the optimum found

• 3-Stages Truss (15 design variables)
• 25 trials
• The pulling back method can derive the
  optimumwith only the small number of island.
• In the difficult problem, the pulling back method
  with the small number of islands can derive the
  better solution than the penalty method.

Islands (Population size) Penalty method Pulling back method
1 (16) 0 24
2 (32) 1 19
4 (64) 4 24
8 (128) 4 19
16 (256) 5 5
32 (512) 12 0
18
Average number of evaluations

• Average number of evaluations required to find
  the optimum.
• 2-Stages Truss (10 design variables)
• The pulling back method can derive the optimum
  with larger number of evaluations as the number
  of islands becomes larger.

Pulling back operation
19
Average number of evaluations

• Average number of evaluations required to find
  the optimum.
• 3-Stages Truss (15 design variables)
• The pulling back method with the small number of
  islands is effective because the pulling back
  operation requires many evaluations.

Pulling back operation
many evaluations are required.
20
Discussion on the comparison of the results

• From the numerical examples,
• The pulling back method with the small number of
  islands can derive the better solution than the
  penalty method.
• In the pulling back method, the small number of
  island is effective.
• The large number of island is not effective
  because the pulling back operation requires many
  evaluations.
• Why is the small number of islands effective?
• Target of the comparison of the result for the
  discussion
• 2-Stages Truss (10 design variables)
• 32 islands and 512 individuals
• Median value of 25 trials

- Rate of the individuals that violates the
constraints - Search mechanism of the both
methods for constraints
21
Rate of the individuals that violates the
constraints
Pulling back operation
The violated constraints are linearized.
Terminal Criteria ofthe pulling back operation
- No violated constraint - Maximum times20 -
Minimum distance1e-8

• In the penalty method, 60 of the population
  violates the constraints.
• In the pulling back method, the invalid
  individuals are only 30.
• More individuals can be used for an efficient
  search than the penalty method.

22
Search mechanism for constraints
In the optimization problem with constraints,the
optimum often exists on the boundary of the
feasible region.

• In the penalty method, the most invalid
  individuals are dead and the population is early
  converged when the population size is small.
• The pulling back operation pulls back these
  individuals near the boundary of the feasible
  region and creates good new search points that
  are different from the parent points.

23
Conclusion

• Structural optimization problem
• Distributed PMBGA(DPMBGA)
• Penalty method and pulling back method
• Comparison of the searching ability
• The pulling back method with the small number of
  island is effective than the penalty method in
  the difficult problem.
• In the pulling back method, The large number of
  island is not effective because the pulling back
  operation requires many evaluations.
• Discussion on the comparison of the result
• The pulling back method can use more individuals
  for an efficient search than the penalty method.
• The pulling back operation can keep the diversity
  of the population even when the population size
  is small.