Nonlinear LeastSquares

1
Non-linear Least-Squares
2
Why non-linear?

• The model is non-linear (e.g. joints, position,
  ..)
• The error function is non-linear

3
Setup

• Let f be a function such that
• where x is a vector of parameters

4
Setup

• Let f be a function such that
• where x is a vector of parameters
• Let ak,bk be a set of measurements/constraints.
  
• We fit f to the data by solving

5
Setup

• Let f be a function such that
• where x is a vector of parameters
• Let ak,bk be a set of measurements/constraints.
  
• We fit f to the data by solving

6
Example
7
Overview

• Existence and uniqueness of minimum
• Steepest-descent
• Newtons method
• Gauss-Newtons method
• Levenberg-Marquardt method

8
A non-linear functionthe Rosenbrock function
Global minimum at (1,1)
9
Existence of minimum

• A local minima is characterized by

10
Existence of minimum
11
Descent algorithm

• Start at an initial position x0
• Until convergence
• Find minimizing step dxk
• xk1xk dxk
• Produce a sequence x0, x1, , xn such that
• 
  f(x0) gt f(x1) gt gt f(xn)

12
Descent algorithm

• Start at an initial position x0
• Until convergence
• Find minimizing step dxk
• using a local approximation of f
• xk1xk dxk
• Produce a sequence x0, x1, , xn such that
• 
  f(x0) gt f(x1) gt gt f(xn)

13
Approximation using Taylor series
14
Approximation using Taylor series
15
Approximation using Taylor series
16
Steepest descent

• Step
• where is chosen such that
• using a line search algorithm

17
(No Transcript)
18
(No Transcript)
19
In the plane of the steepest descent direction
20
Rosenbrock function (1000 iterations)
21
Newtons method

• Step second order approximation
• At the minimum of N

22
(No Transcript)
23
Problem

• If is not positive semi-definite, then
  
• is not a descent direction the step increases
  the error
• function
• Uses positive semi-definite approximation of
  Hessian based on the jacobian
• (quasi-Newton methods)

24
Gauss-Newton method

• Step use
• with the approximate hessian
• Advantages
• No second order derivatives
• is positive semi-definite

25
Rosenbrock function (48 evaluations)
26
Levenberg-Marquardt algorithm

• Blends Steepest descent and Gauss-Newton
• At each step solve, for the descent direction

27
Managing the damping parameter

• General approach
• If step fails, increase damping until step is
  successful
• If step succeeds, decrease damping to take larger
  step
• Improved damping

28
Rosenbrock function (90 evaluations)
29
Resynthesizing Facial Animation through 3D
Model-Based Tracking, Pighin etal.,
??

??

??

??

??
Video frame
Model
30
Model fitting
??
2

??

-
??

??

??
Video frame
Model
31
Model fitting
??
2

??

-
??

??

??
Video frame
Fitted model
Model
32
Resynthesizing Facial Animation through 3D
Model-Based Tracking, Pighin etal.,

• Facial tracking by fitting face model

33
Resynthesizing Facial Animation through 3D
Model-Based Tracking, Pighin etal.,

• Facial tracking by fitting face model
• Problem
• Given an image I
• A 3D face model

34
Resynthesizing Facial Animation through 3D
Model-Based Tracking, Pighin etal.,

• Facial tracking by fitting face model
• Problem
• Given an image I
• A 3D face model
• Minimize
• Where is the image produced by the face
  model
• And is a regularization term.

35
Problem setup
36
Solving

• Using a gaussian pyramid
• of the input image
• For level in coarserLevel finerLever
• Initialize using solution at level-1
• Minimize at level using Levenberg-Marquardt
• Solution is solution at finerLevel

37
Tracking results
38
Conclusion

• Get a good initial guess
• Prediction (temporal/spatial coherency)
• Partial solve
• Prior knowledge
• Levenberg-Marquardt is your friend