9 Unconstrained minimization 9.7 Implementation

1
9 Unconstrained minimization9.7 Implementation

• Pre-computation for line search
• To speed up the line search process
• Computing the Newton step
• To dominate the work involved in the line search

Edited by amylym_at_gmail.com Lee, Young Min
Visual Information Laboratory at Sogang
University
2
9.7.1 Pre-computation for line search
Composition with an affine function

• Simple method

cost

• More efficient method

Flop (662p)We define a flop as one addition,
subtraction, multiplication, or division of two
floating-point numbers. To evaluate the
complexity of an algorithm, we count the total
number of flops, express it as a function
(usually a polynomial) of the dimensions of the
matrices and vectors involved.
3
9.7.1 Pre-computation for line search
4p
4pn
4
9.7.1 Pre-computation for line search

• Simple method

Total cost

• More efficient method

Total cost
5
9.7.2 Computing the Newton step
Newton system / normal equation The work of
computing the Newton step dominates the work
involved in the line search.
Cholesky factorization of H To compute L that
satisfies
6
Appendix C.3.2 Cholesky factorization
http//en.wikipedia.org/wiki/Cholesky_factorizatio
n The Cholesky algorithm, used to calculate the
decomposition matrix L, is a modified version of
Gaussian elimination.
7
(No Transcript)
8
9.7.2 Computing the Newton step
9
9.7.2 Computing the Newton step
10
9.7.2 Computing the Newton step
11
9.7.2 Computing the Newton step