PowerPoint-Pr

1
Lecture on Numerical Analysis
Dr.-Ing. Michael Dumbser
08 / 11 / 2007
2
Numerical Solution of Nonlinear Equations
Task compute approximately the solution x of the
nonlinear algebraic equation
(1)
If the problem is not in form (1), then it can
always be cast into form (1) by putting all terms
on the left hand side. Examples
3
Numerical Solution of Nonlinear Equations
Task compute approximately the solution x of the
nonlinear algebraic equation
(1)

• Two problems present even at the analytical
  level
• For general functions f(x), there may be no
  solution at all. Example
• For general functions f(x), there may be more
  than one solution. Example

4
The Regula Falsi Method
f
x
x0
x2
x3
x1
x4
(I) Before we can start using this method, we
must guess two points x0 and x1 where the
function f(x) has opposite sign, i.e.
(II) Now connect the two points (x0, f(x0)) and
(x1, f(x1)) with a straight line and compute the
intersection of this line with the x-axis,
denoted by xi. The point xi is given by
(III) Compute the sign of f(x2) and compare
against the signs of f(x0) and f(x1). Then update
the interval boundaries according to the
result, i.e. set
If f(xi) lt tolerance then STOP (in Maple
break), otherwise go back to step (II).
5
The Regula Falsi Method
f
x
x0
x2
x3
x1
x4
Advantages - it is guaranteed that the method
finds a solution of f(x)0 because we already
know that it the solution
is contained in the initial interval x0x1.
- The method is very robust and reliable
Disadvantage - it can not be applied if we do
not have an initial guess of the interval x0x1.
6
The Secant Method
f
x
x0
x2
x1
x3

• Variant of the regula falsi method we start
  from any two points x0 and x1

• Advantage no initial interval must be guessed
• Disadvantage may not always converge towards a
  solution !

7
The Secant Method
f
x
x0
x1

• Variant of the regula falsi method we start
  from any two points x0 and x1

• Advantage no initial interval must be guessed
• Disadvantage may not always converge towards a
  solution !

8
Newtons Method
f
x
x2
x0
x1

• Differential form of the secant method. Instead
  of using the secant, Newton proposes to use the
  tangent!

• Advantages - no initial interval must be
  guessed - very fast
• Disadvantage may not always converge towards a
  solution !

9
Newtons Method
f
x
x2
x0
x1

• Differential form of the secant method. Instead
  of using the secant, Newton proposes to use the
  tangent!

1. If f(xi) lt tolerance then STOP (in
   Maple/MATLAB break) or
2. If Dx lt tolerance then STOP

10
Exercise 1
Write a Maple worksheet and a MATLAB script that
solve the nonlinear equation using the regula
falsi method. Choose an appropriate starting
interval.
Exercise 2
Now write a MATLAB script that solves the problem
of exercise 1 using the Newton method. The user
should have the possibility to choose the
starting guess, the maximum number of iterations
as well as the tolerance.
11
Globally Convergent Newton Methods
From the previous Example 4 we find that even for
strictly monotone functions f(x) Newtons method
may not always converge from arbitrary starting
points x0, but only from starting points that are
sufficiently close to the solution of the
problem. However, if the starting point is
sufficiently close, the method converges. We
therefore say the original Newton method is only
locally convergent. The local convergence
behaviour of Newtons method and the possible
failure to converge for initial guesses far away
from the solution can be mainly explained by too
large step sizes Dx
12
Globally Convergent Newton Methods
As seen in the sketch of the previous slide, the
absolute value of the function f(x) slightly
increased during one Newton step. To avoid this
problem, we therefore add the following
requirement The function f(x) must always
decrease during one Newton step, i.e.
If this is not fulfilled, the step size is
sucessively reduced until the criterion is
satisfied. The globally convergent Newton method
then reads as follows
DO i0,1,
Free parameter. Choose e.g. t 0.5
WHILE
END
UNTIL
13
Globally Convergent Newton Methods
Exercise 5
Write a modified MATLAB script GlobalNewton.m to
solve the nonlinear equation with the
globally convergent Newton method.

1. Use x0 2 as initial guess.
2. Use x0 2.5 as initial guess. What do you
   realize?
3. Use x0 106 as initial guess. Does the method
   converge?

14
Computation of the Roots of a Polynomial
We now apply Newtons method to a special case of
nonlinear algebraic equation where thefunction
f(x) is a polynomial of degree n
The values x which satisfy eqn. (1) are called
the roots of the polynomial pn(x). We suppose in
the following that all the roots of the
polynomial are distinct and real.
Suppose x x is a root of pn(x), i.e. then
we have If we knew all the roots xi we would
have the following expression of the polynomial
15
Computation of the Roots of a Polynomial
A simple and straightforward application of
Newtons method now consists in finding a
first root x1 (typically the largest one) of the
polynomial pn and then compute the largest root
x2 of the polynomial pn-1 obtained by dividing
the polynomial pn by ( x x )
After having found the root x2 of pn-1 we divide
again
and compute the next root x3 and so on, until all
the n roots of pn have been found. In general,
we have
16
Computation of the Roots of a Polynomial
To express the polynomial pn-j(x) and its
derivative, two simple MATLAB loops defining the
auxiliary variables Dj and Sj are necessary
with
We then obtain the so-called Newton-Maehly method
17
Exercise 3
(1) Write a MATLAB script PolyRoot.m that finds
all the roots of a general polynomial of
degree n. The user should be able to
specify the coefficients a of the polynomial.
Then, two MATLAB function files, called
Polynomial.m and DPolynomial.m compute the
polynomial and its first derivative in point x
using the coefficients a. Further user
inputs required are tol, NMAX and x0. (2)
Apply your general MATLAB script to the
particular polynomial (3) Now try your MATLAB
program on the polynomial Hint Use
Maple to rewrite this polynomial in standard form
18
Efficient Evaluation of Polynomials
The standard format of defining a polynomial is
computationally not the most efficient way
toevaluate a polynomial
In this example, we have 4 additions and 10
multiplications
A computationally more efficient but
mathematically fully equivalent way of writing
the polynomial p(x) is the so called Horner
scheme, which uses the distributive law
Now, we still have 4 additions but only 4
multiplications!
19
Globally Convergent Newton Methods
Exercise 4
Now try your original MATLAB script Newton.m to
solve the nonlinear equation

1. Use x0 2 as initial guess.
2. Use x0 2.5 as initial guess. What do you
   realize?

20
Newtons Method for Systems of Equations
The extension of Newtons method to systems of
nonlinear algebraic equations with several
unknown variables is straightforward.
The problem is written in standard form in
matrix-vector notation as follows
Linearization (multi-dimensional Taylor series up
to first order) leads to
The matrix D is called the Jacobian matrix of the
vector function f with respect to ist vector
argument x and is the multi-dimensional
equivalent to the first derivative of a scalar
function.
This corresponds to solving the linear system

1. If f(xi) lt tolerance then STOP (in
   Maple/MATLAB break) or
2. If Dx lt tolerance then STOP

i.e. the vector norm of f is the so-called
function residual.
21
Exercise 6
Write a MATLAB script NewtonSys.m that solves the
nonlinear system of equations using the
Newton method for systems. The user should have
the possibility to choose the initial guess for
the vector of unknowns, the maximum number of
iterations as well as the tolerance. The vector
function and its Jacobian matrix should be
implemented in two separate .m files, called
FuncSys.m and DFuncSys.m.
22
Globally Convergent Newtons Method for Systems
The extension of the globally convergent Newtons
method to systems of nonlinear algebraic
equations with several unknown variables is done
as in the scalar case
DO i0,1,
WHILE
END
UNTIL
23
Exercise 7
We want to solve the following nonlinear system
of equations (1) Use the classical Newton
method for systems starting from 0,0 (2) Use
the classical Newton method for systems starting
from 1000,1000 (3) Can you find a starting
point from which the classical Newton method
converges? (4) Construct a globally convergent
Newton method for systems and start again
from the points 0,0 and 1000,1000. Does the
global method even converge from the
starting point 1e6,1e6 ?