MATHEMATICAL OPTIMIZATION METHODS

1
MATHEMATICAL OPTIMIZATION METHODS i.e. how to
optimize a system through definition of
mathematical objective functions
2
(No Transcript)
3
Optimization

• The design concept has been selected, reasonably
  well defined, and some sort of model of the
  system has been developed.
• Optimization is about selecting the design
  parameters that optimize some performance measure
  (a state variable).
• The performance measure is referred to as the
  objective function.

4
Systems optimization in nature Atomic
positions in engineering materials (strength,
stiffness, stability etc.) Liquid droplet in
zero gravity (spherical i.e. the smallest
area/volume ratio) Tall trees with ribs at their
bases Beehive honeycombs Genetic mutation for
survival etc.
5
Engineering systems optimization a dynamic
process because engineering systems are
dynamic! Numerical optimization methods often
the most flexible way assumes that the system
logic is completely understood
6
In this course Introductory concepts chapter
1 1D unconstrained optimization chapter
2 Unconstrained optimization chapter
3 Constrained optimization chapter 5
7
Historically first optimization method
gradient method (Cauchy 1847) i.e. objective
functions (performance measures) plotted, then
defined as optimized when dy/dx 0 similar to
Taguchi plots! Sometimes mathematical methods
more appropriate than experimental design methods
8
1D Unconstrained Optimization
Objective function y f(x)
f(x) is optimized when the slope of y with
respect to x is zero.
Necessary condition for optimization
9
1D Unconstrained Optimization
Simple Example f(x) -x2 4x -2
10
An example to begin A mill manufactures fabrics
of different colours. It has a high quality dying
unit, but with limited capacity. Therefore it
must subcontract some of its dying operations. At
the end of the day, the mills profit must be
maximized In this system profit ( total price
total cost) is to be maximized For this
system r total fabric quantity that can be
dyed in-house R total fabric quantity that can
be dyed by a subcontractor nf number of
different coloured fabrics xi quantity of
fabric i dyed in-house yi quantity of fabric i
dyed by subcontractor Pi unit price ci unit
cost to dye in-house si unit cost to dye using
subcontractor di amount of fabric that needs to
be dyed per time interval
11
Objective function maximize profit i.e.
maximize subject to design variables xi and
yi i.e. the subject of the analysis let
nf4, r50 and R1000 (i.e. unlimited) and d125,
d245, d350, d460 (Pi-ci)10.0, 6.5, 6.0,
5.0, (Pi-si)-2.0, 5.5, 5.0,4.0
12
Solution xi25,0,0,25 yi0,45,50,35 pro
fit 1012.50 found computationally using simple
mathematical techniques - classic system
optimization example - can be solved using linear
programming - a constrained example
13
1D Unconstrained Optimization
Setting the derivative equal to zero only tells
us that we have found a point where the slope is
zero, not whether this point is a maximum,
minimum, or an inflection point. Therefore, it
is a necessary, but not sufficient condition for
optimality.
14
1D Unconstrained Optimization
Maximum point
Minimum point
Inflection point
15
1D Unconstrained Optimization
Example
16
1D Unconstrained Optimization
Example
17
Mathematical fundamentals
Linear Independence of vectors Let Xk, k1,2,
. ,m be a set of m vectors The vectors are
linearly independent if the linear combination
implies that all
18
Example

• Determine if the following 3 vectors are linearly
  independent

Answer Calculate the determinant. Since the det.
A 0, it does not follow that all have
to be zero
19
or Aa0 to check calculate the
determinant if det A ?0 the vectors are
linearly independent here det A 0 so vectors
are linearly dependent det A a11A11 a12A12
a13A13 (for a 3x3 matrix)
20
Quadratic forms and positive definite
matrices Consider the function f x12 6x1x2
9x22 in matrix notation this is If A is
symmetric we have a quadratic form A quadratic
form is positive definite if xTAx gt 0 for every
non-zero vector x
21
What is positive definiteness? A matrix A is
positive definite (PD) if all of its eigenvalues
are strictly positive Eigenvalues? read
up! for Ax ?x ? is an eigenvalue (scalar) and
x is an eigenvector the solution to det(A - ?I)
0 gives the eigenvalues A then det(A -
?I) 0 i.e.
22
Sylvesters test for Positive Definiteness Let
Ai be the submatrix formed by deleting the last
(n-i) rows and columns from the nxn matrix
A Then, A is positive definite if det (Ai) gt 0
for i1,2,., n What is positive definiteness?
23
Example Determine if this quadratic form is
positive definite f 2x12 5x22 3x32
2x1x2 4x2x3 when written in matrix form
this becomes using Sylvesters test check the
determinant for each sub-matrix 2 gt 0 10 (1)
9 gt 0 2(11) 1(-3) 0 19 gt 0 so yes it is!
24
Sets xx 1 - example of a closed
set xx lt1 - example of an open set A set is
bounded if it is contained within some sphere of
finite radius i.e. for any point a in the set aTa
lt c where c is a finite number A set is compact
if it is closed and bounded The set of real
numbers in one dimension R or R1 In
n-dimensions, the real space Rn The
neighbourhood of a point c described by
x??x-c??ltd for some dgt0
25
Convexity A set S is called a convex set if for
any two points in the set, every point on the
line joining the two points is in the
set convex set non-convex set
26
(No Transcript)
27
(No Transcript)
28
(No Transcript)
29
(No Transcript)
30
Minimum and maximum value theorem (the
Weierstrass Theorem) Let f be a continuous
function defined over a compact set. Then, there
are points x and x in the set, where f attains
its minimum and maximum respectively. That is,
f(x) and f(x) are the minimum and maximum
values of f in the set
31
Gradient vector Consider an objective function
with more than one dependent variable f(x)
f(x1, x2, , xn) gradient given by a column
vector necessary condition for optimality
is ?f (x) 0 x is called a stationary point
32
Gradient of a vector of m functions
Once ?f is known, the derivative of f in any
direction s at a point c, termed the directional
derivative is obtained as
33
(No Transcript)
34
Taylors theorem used to derive optimality
conditions etc. by constructing linear and
quadratic approximations to non-linear functions
35
Example
36
Every analytic function can be represented by a
Taylor series very important in complex
analysis critical in the definition of
mathematical convergence