Optimisation and Least Squares Fitting

1
Lecture 4

• Optimisation and Least Squares Fitting

2
A reminder
3
Parameter estimation (SSP 2.3.1)
A sequence of values of a random variable,
x1,x2, . xN how to estimate the parameters of
the distribution they come from
4
Uncertainty in the estimate m
The estimate of the mean, m, is itself a random
variable, i.e. different sequences x1,x2, . xN
would give rise to different values. The
standard deviation of this distribution which
is a measure of the uncertainty in m is given by
(Lecture 2)
5
Optimisation

• Problem is to find the parameters, or
  configuration which will minimise a specified
  cost function U(a), where there may be several
  parameters ai and the surface U(a) may be very
  complicated.

6
U(x,y)Cos(py)Sin3(py)
7
Fitting a function to a set of data points
Example Fitting a straight line
8
(No Transcript)
9
Straight line fit 1
10
Straight line fit 2
11
Which line is best?
12
What criterion to use?
Method of Least Squares
Choose the parameter set ai which minimises
13
What to use for the weights wi?
More accurately determined points should be given
greater weight
The error on each point, based on ni
measurements, has standard error ?i proportional
to 1/?ni
14
General Least Squares Fit
15
Linear Least squares fit (CTP1.5)
Fitting with a function linear in m parameters
16
If we can solve the system of equations
we have Linear Least Squares

• If direct solution not possible,
• Analytical algorithm, e.g. Marquardt algorithm
• Heuristic methods

17
Linear Least Squares (CTP 1.5)
Array of coefficients obey set of linear equations
18
Baz,
aB-1z
19
General result Combination of variates
20
(No Transcript)
21
Uncertainties in parameters a
Variance on parameter aa
Covariance between aa and ab
22
Fitting with orthogonal functions
23
Example
24
(No Transcript)
25
Including the errors
26
red including the errorsblue not including the
errors
27
U(a1,a2)
28
Non-linear Least squares fit (CTP2.2, and class
notes)
Fitting with a function linear in m parameters
29
Parameters occur in non-linear form, Examples
30

• Combination of
• Steepest descent, and
• Parabolic approximation

31
Typical surface for 2 parameters
32
Steepest descent
33
Steepest descent
Start at arbitrary a(i) and move against the
local gradient
34
Parabolic approximation
35
Marquardt algorithm (see Notes distributed)
Combine steepest descent and parabolic
approximation interactively. Write