Kriging Based Modeling of Pareto Optimal Front in Multi Criteria Optimization

1
Kriging Based Modeling of Pareto Optimal Front in
Multi Criteria Optimization

• Under the guidance of Prof. K. Sudhakar and Prof.
  P. M. Mujumdar

M.Tech Defense Devendra Kumar Rane
2
Outline

• Introduction
• Basic Concepts and Definitions
• Formulation methods and algorithms
• Gradient based algorithms
• Evolutionary algorithms
• Motivation
• Proposed algorithm
• Results and summary

3
Multi-Criteria Problem

• Problem Definition

Find
To Minimize
Subjected to
4
Multi-Criteria Optimization
5
Basic Concepts and Definitions

• Pareto Optimality
• Utopia Point

• Preference function
• Convexity of Multi Criteria Problem
• The components of the objective function vector
  f(x) are convex.
• The components of the vector of the inequality
  constraints g(x) are convex.
• The components of the vector of the equality
  constraints h(x) are affine-linear functions of x.

6
Problem Formulation

• Method of Objective weighting
• Method of Distance Functions
• Demand Level vector
• Trade-off Method
• with
• Min-Max Formulation
• where
• are the known minima of individual functions

7
DMs flowchart

• Preference Function approach

8
DMs flowchart

• Physical Programming Approach

9
Physical Programming

• Physical Programming Lexicon

10
Physical Programming

• Physical Programming Lexicon

11
Physical Programming

• Physical Programming Lexicon

12
Physical Programming

• Physical Programming Lexicon

13
Physical Programming

• Pareto-sensitivity analysis

• Algorithm
• Obtain the Jacobian of all the active
  constraints.
• Get the Projection matrix to the tangent
  hyper-plane of the active constraint sets at a
  given Pareto-optimal point
• Get the feasible direction for maximum
  improvement in an objective
• Calculate derivatives of one objective w.r.t the
  others and form a trade-off matrix T.
• Trade-off matrix All diagonals one, and at least
  one element in the row, negative, for
  Pareto-optimality.

14
Physical Programming

• Pareto-surface approximation

• Algorithm
• The Pareto-optimal front can be approximated by
  using the Hessian and first-order terms by the
  elements of the Trade-off matrix.
• The data for the Hessian is generated by the
  Pareto-data generation step.

15
Physical Programming

• Pareto-data generation

• Algorithm
• Predictor step Using the first order
  approximations of f,g and h, the compromise
  problem is solved to get the design vector
  corresponding to the aspiration point.
• Corrector step With bound and active constraint
  switch considerations, project design point to
  objective space using actual functions and not
  the linear approximations.
• Iteratively, a approximated Pareto-front can be
  generated near a point.

16
Physical Programming

• Pareto-data generation

17
Evolutionary Algorithms

• Similar to algorithms for single objective
• Initialization
• Fitness Function
• Based on Domination
• Domination based sorting
• Selection
• Crossover
• Mutation

18
Motivation

• Genetic Algorithms Computationally intensive.
• Gradient Based Pareto front capturing is
  inefficient even with parameter adjustments.
• Genetic Algorithms Niche solutions.
• Gradient Based Dependency on initialization.
• Need for efficient modeling of Pareto optimal
  front

19
Proposed Algorithm

• Simple Kriging is inefficient to capture Pareto-
  optimal with very less data points
• Use of Pareto sensitivity analysis approach in
  Physical programming to generate gradients
• Apply gradient enhanced kriging techniques to
  approximate model
• Helpful for better approximation and visualization

20
Test Case-1
Maximize
Maximize
Subject to
21
Test Case-1
22
Test Case-1
23
Test Case-1
24
Test Case-1
25
Test Case-2
Maximize
Maximize
where

Subject to
26
Test Case-2
27
Test Case-2
28
Test Case-2
29
Test Case-2
30
Test Case-3
Minimize
Minimize
Minimize

Subject to

31
Test Case-3
Gradient enhanced kriging approximation
Simple kriging approximation
Actual pareto-front
32
Conclusion

• Gradient enhancement provides better results.
• Can be used by the Decision Maker to estimate
  near pareto-optimal solutions.
• Can be used for initialization of algorithms to
  get exact solution with lesser computations.
• Niching problems in genetic algorithms can be
  coped up with using proposed post processing.

33
Thank you
34
Kriging (reference notes)

• Problem of predicting value at specified point
  given the observations at some points in field
• f(x) ? ?i y(xi)
• ?i depend on the distance of the test point x
  from observed points (spatial interpolation

35
Kriging model (reference notes)

• Y(x) ? ?i fi(x) Z(x)
• ? ?i fi(x) represents regression model
• Z(x) is the stochastic process
• E(Z(x)) 0
• Cov(Z(w), Z(x)) ?z2 R(w, x)
• ?z2 is called process variance
• R(w, x) is the spatial correlation function
• SCF is function of axial distance between two
  points (wj xj)

36
DACE Model (reference notes)

• Observations y y1 yn for x x1 xn
• Y(x) F? Z(x)
• Cov(Z(w), Z(x)) ?z2 R(w, x)
• R(w, x) ? exp( - ?j wj - xjpj )
• Unknowns to be determined
• ?i i1,m
• ?z2
• ?j and pj j1,d

37
MLE (reference notes)

• Given ?j and pj
• MLE of ?
• ? (FT R-1 F)-1 FT R-1 y
• MLE of ?z
• ?z2 1/n (y F?)T R-1 (y F?)
• Log Likelihood is
• -1/2 n ln(?z2) ln R

38
DACE predictor (reference notes)

• Best Linear Unbiased Predictor (BLUP) at an
  untried x is
• Y(x) F? rT(x) R-1 (y F?)
• where rT(x) R(x1,x) R(xn,x) is the
  vector of correlations between the Zs at the
  design points and x
• ?j and pj are determined by maximizing the
  likelihood by numerical optimization.

39
DACE model Estimation

• Start with initial guess of ?j and pj j1,d
• Repeat to minimize log likelihood function
• Compute R and R
• Compute MLE of ?z
• Compute log likelihood function
• Update values of ?j and pj
• (any optimization method can be used to solve
  this unconstrained minimization problem)
• Compute MLE of ?i

40
DACE predicted error

• The correlation of the errors also affect
  estimate of prediction accuracy. If x is closer
  to any of the observed points, we can be much
  more confident in our prediction of Y(x) than if
  x were far away from all sampled points.
• Mean Squared Error (MSE)