Evolutionary Computational Intelligence

1
Evolutionary Computational Intelligence

• Lecture 10a Surrogate Assisted

Ferrante Neri University of Jyväskylä
2
Computationally expensive problems

• Optimization problems can be computationally
  expensive because of two reasons
• high cardinality decision space (usually
  combinatorial)
• computationally expensive fitness function (e.g.
  design of on-line electric drives)

3
High Cardinality Decision Space

• Under such conditions it should be tried, on the
  basis of the application, to reduce the
  cardinality by means of an a priori analysis or
  an heuristic to detect a promising region of the
  decision space
• Memetic approach (e.g. intelligent initial
  sampling) can be beneficial

4
Computationally expensive fitness

• It might happen that the fitness function
  evaluation requires by itself a lot of
  computational effort (e.g. in online PMSM drive
  design each fitness evaluation requires 8 s)
• In such conditions it should be found a way to
  reduce the numer of fitness evaluations and still
  reach the optimum

5
Surrogate Assisted Algorithms

• Surrogate Assisted Algorithms employ approximated
  models of the fitness function (cheap)
  alternatively with the real fitness (expensive)
• One of the crucial problems is the model to be
  employed and how to arrange such a combination

6
Global vs Local Surrogate models

• There are two complementary and contrasting
  algorithmic philosophy
• Global Surrogate Models attempt of finding an
  approximated model of the landscape over all the
  decision space
• Local Surrogate Models attempt of approximating
  locally the landscape over the neighborhood of a
  certain point

7
Comparison between the two philosophies

• Global models assume that a wide knowledge of the
  decision space allows to build up an accurate
  model that can be employed as a cheap alternative
  of the real fitness
• Local models assume that a huge amount of
  information does not help in determining an
  accurate model and thus it is preferable to build
  up models that approximate only locally the
  behavior of the landscape
• Global models employ one very complex model,
  Local models employ many simple approximated
  functions

8
Coordination of models/real fitness

• The right way to perform the coordination is very
  problem dependent, both deterministic and
  stocastic rules have been implemented
• Models can be installed in both evolutionary
  framework and local searchers

9
Surrogate Assisted Hooke-Jeeves Algorithm

• Surrogate Assisted Hooke Jeeves Algorithm
  (SAHJA) deterministic scheme for coordinating
  real fitness and a linear model obtained by least
  square method
• Computes N1 points and generates a local linear
  model for calculating the remaining N points
  (Cost of exploratory move is thus kept constant)
• Check every directional move, by calculating the
  real fitness if a surrogate was prevously
  calculated (does not allow search directions by
  means of surrogate points)

10
SAHJA
11
SAHJA Results

• Very promising algorithmic performance
• Noise filtering

12
Evolutionary Computational Intelligence

• Lecture 10b Experimentalism

Ferrante Neri University of Jyväskylä
13
Goals

• To propose a research protocol in order to
  execute a fair experimental comparison which
  allows us to check whether the newly proposed
  algorithm outperforms the methods existing in
  literature
• In other words, if I designed a novel algorithm
  how can I be sure that my work outperform (for a
  certain problem) the state of the art?

14
Towards Performance Comparison

• If I designed a novel algorithm B how can I prove
  that B outperforms the benchmark algorithm A? How
  can I thus have a confirmation that the novel
  algorithmic component is really effective?
• Performance is an abstract concept not related to
  a specific machine. It is the capability of an
  algorithm of reaching a good performing solution
  in a certain time interval. The time trigger is
  the number of fitness (functional) evaluations

15
Experimental Setup

• For both A and B, a certain number n of runs must
  be performed
• The average best fitness values (e.g. at the end
  of each generation) must be saved
• N.B. for making the trends comparable an
  interpolation can be necessary
• Standard deviation bars can also be included

16
Two Possible kinds of outperforming

• Case 1
• A and B converge on different final values
• Case 2
• A and B converge on the same final values but
  with different convergence velocities

17
Case 1

• The data define two Tolerance Intervals (TIs)
• It is fixed a desired confidence level d
• The proportion ? of a set of data which falls
  within a given interval with a given confidence
  level d are determined by
• ? 1 -a/n
• where n is the number of available samples and
  a is the positive root of the equation
• (1 a) - (1 - d) ea 0

18
Case 2

• A threshold value fthr is fixed. If during an
  experiment fbest lt fthr then the algorithm
  almost converged
• For the n experiments, the number of fitness
  evaluations necessary to verify the inequality
  fbest lt fthr defines a TI
• The probability ? that an algorithm requires no
  more fitness evaluations than the most unlucky
  case is given by
• ? 1 -d/n
• where n is the number of the available
  experiments.
• d is given by
• d -ln(1 - d)

19
How to conclude

• In both the cases, if the tolerance intervals are
  not separated it is impossible to establish that
  B outperforms A in all the cases. In this case it
  is possible only to state that B outperforms A in
  average