Nonlinear function minimization Fish 458

1
Nonlinear function minimizationFish 458

2
Readings

• Ecological Detective, chapter 11

3
References

• Ecological Detective chapter 11
• Hilborn and Walters
• Bard Nonlinear Parameter Estimation

4
The general problem

• to find a maximum or minimum in a
  multi-dimensional surface

5
Warning

• The material in this lecture looks complex and
  very scary
• In practice the action is in intuition and good
  practice
• Much of the algebra is presented for
  completeness, and we wont deal with it outside
  the lecture
• You can master this subject!

6
How to find the minimum sum of squares

• direct search (ok for 1-3 parameters)
• algebra (linear regression - linear models)
• non-linear gradient searches

7
Hill climbing in reverse

• Conceptually we want to walk down hill until
  every direction is up - to minimize sum of
  squares

8
(No Transcript)
9
(No Transcript)
10
(No Transcript)
11
How to avoid false summits

• Make sure you are on a true summit
• Look in all directions
• In SOLVER restart from solution to make sure it
  has converged
• See if other starting points end up at the same
  place
• Start SOLVER from a number of places, and see if
  they all go to the same place

12
Multiple starting points
13
(No Transcript)
14
Basic approach

• Start with a guess about X
• See which direction is down (calculate the slope
  (dy/dx)
• Look at the slope and the change in slope (first
  and 2nd derivatives) to guess about how far we
  have to go until we reach the bottom
• Move that direction
• Start over again

15
Newtons methodYes that is Sir Isaac
NewtonEcological Detective pp 267
16
Basic logic

• Want to set first derivative to zero
• 2nd derivative is rate of change of first
  derivative
• divide first derivative by second derivative to
  get number of units of x to jump to find where
  first derivative is zero
• set lambda lt 1 to prevent overshooting

17
Basic theory

• If the curve is quadratic (as it is in linear
  models), then the 2nd derivative is uniform over
  the entire range of X and you can jump right to
  the minimum
• BUT if the curve is not quadratic, then you have
  to iterate

18
Demo in Newtons method.xls
19
Linear models formulaHilborn and Walters p207
20
Generalized nonlinear minimization (Hilborn and
Walters page 213)
The Jacobian
This is analogous to the X matrix in linear models
Here the sensitivity of Y to b is
21
Multidimensional Newtons
22
Other approaches

• Derivative free - the SIMPLEX method
• This is a very sophisticated form of hill
  climbing
• Simulated annealing
• Randomly jump to a new spot, if it is better then
  stay there, if it is worse, go back to initial
  jump

23
Parameter confounding and correlation
The Hessian
24
Numerical approximation
For the diagonal
25
For the off-diagonal elements
26
(No Transcript)
27
Fraser Chum sum of squares contour
28
To add

• Mle and cv for each parameter

29
Variance covariance matrix
30
Fraser Chums Variance-Covariance matrix A
31
Parameter correlation matrixsee Hilborn and
Walters page 208
32
Fraser Chums Correlation matrix C
33
Further complications

• Parameter confounding
• Problems with numerical derivatives
• Non continuous problems
• Integer parameters
• Multiple minima
• Constrained parameters

34
Constrained parameters

• Transform bounded parameters to unbounded using

Show demo atan_demo.xls
35
Hints for successful fitting

• Test your code by fitting known data
• Find good fit to the data as a starting point
• Graph data and fit
• Manually change models to get good fit
• Check for convergence
• Restart from final solution
• Restart from different starting points

36
More hints

• Constrain population sizes to not go negative
• Bounded parameters pose problems
• Usually better to do the bounding in your code or
  spreadsheet ABS and ATAN methods
• Particularly problematic are multiple proportions
  that must add to 1
• Fix each p to be 1-sum of the previous ones
• In SOLVER use automatic scaling and set
  convergence criteria smaller

37
Conclusions

• Is as much art as science
• You cannot just plug numbers into a program and
  hope for the best -- you must make checks to
  assure convergence
• The demise of a promising shark analyst
• Takes time and experience - but is well rewarded