Sin t

1
EngineeringComputation

• Lecture 2

2
Round-off Error due to Arithmetic Operations
Smearing Occurs when individual terms are
larger than summation itself. Consider the
exponential series with x -10 Consider
formulas such as
With 7-decimal-digit accuracy exact answer
4.54 10-05 computed answer
6.26 10-05 (45 terms) wrong sign
! Largest intermediate terms are 9th
2,755.732 10th 2,755.732
3
Truncation Error
Error caused by the nature of the numerical
technique employed to approximate the
solution. Example Maclaurin series expansion of
ex
If we use a truncated version of the series
Then the Truncation Error is
4
Approximations and Rounding Errors

• Precautions
• Sums of large and small numbers due to equaling
  the exponent. They are common in sums of infinite
  series where the individual terms are very small
  when compared with the accumulated sum. This
  error can be reduced by summing first the small
  terms and using double precision.
• Cancellation of the subtraction The subtraction
  of very similar numbers.
• Smearing The individual terms are larger than
  the total sum.
• Inner products They are prone to rounding
  errors. Thus, it is convenient to use double
  precision in this type of calculations.

5
Error Propagation Errors which appear because
we are basing current calculations on previous
calculations which also incurred some form of
error Stability and Condition
Number Numerically Unstable Computations which
are so sensitive to round-off errors that errors
grow uncontrollably during calculations. Condition
sensitivity to such uncertainty "well
conditioned" vs. "ill conditioned" Condition
Number measure of the condition i.e., extent
to which uncertainty in x is amplified by
ƒ(x) C.N. ? 1 gt "well-conditioned"
C.N. gtgt 1 gt "ill-conditioned"
6
Taylor Series Expansion
Basic Idea Predict the value of a function,
ƒ, at a point xi1 based on the value of the
function and all of its derivatives, ƒ, ƒ', ƒ",
at a neighboring point xi Given xi, ƒ(xi),
ƒ'(xi), ƒ"(xi), ... ƒn1(xi), we can predict
or approximate ƒ(xi1)
7
Taylor Series Expansion
General Form
h "step size" xi1 xi Rn remainder
to account for all other terms
with xi ? ? ? xi1
O (hn1) with x not exactly known "on the order
of hn1 " Note f(x) must be a function with n1
continuous derivatives
8
Taylor Series Expansion
? 0th order T.S. approx. (n 0) f(xi1)
f(xi) O (h1) ? 1st order T.S. approx. (n
1) f(xi1) f(xi) hf '(xi) O (h2) ? 2nd
order T.S. approx. (n 2)
? nth order T.S. approximation will be exact for
an nth order polynomial
9
Taylor Series Expansion
f(x )
f(xi )
Zero order
f(xi1 )? f(xi )
First order
f(xi1 )? f(xi )f '(xi )h
Second order
f(xi1 )? f(xi )f '(xi )h )f "(xi )h2/2!
True
f(xi1 )
x
xi
xi1
h
10
Numerical Differentiation from Taylor Series
Expansion
Objective Evaluate the derivatives of function,
ƒ(xi), without doing it analytically. When would
we want to do this? 1. function is too
complicated to differentiate analytically
2. function is not defined by an equation,
i.e., given a set of data points (xi, ƒ(xi)),
i1,,n i 0 1 2 3 4 xi 1.0 3.0 5.0 7.0 9.0
ƒ(xi) 2.3 4.1 5.5 5.7 5.9
11
Numerical Differentiation from Taylor Series
Expansion

• First derivative with backward difference.

12
Numerical Differentiation from Taylor Series
Expansion
Backward Difference Approx.  First Derivative
Letting h xi - xi-1
first backward difference
13
Example of 1st Backward FDD
Using data below calculate ƒ'(x1)
i 0 1 2 3 4 xi 1.0 3.0 5.0 7.0 9.0
ƒ(xi) 2.3 4.1 5.5 5.7 5.9 First Backward
Finite-Divided-Difference at x1
f ' (x1) ? 0.9 O (h)
14
Backward Difference Approximation
Second Derivative
O(xi-2 xi3) with h xi xi-1 and
2h xi xi-2 The 2nd order approximation to
ƒ(xi-2) becomes ƒ(xi-2) ƒ(xi) 2hƒ'(xi)
2h2 ƒ"(xi) O (h3) 1 2nd order
approximation to ƒ(xi-1)
2
15
Backward Difference Approximation
Subtracting 22 from 1 yields f(xi-2)
2f(xi-1) f(xi) h2f"(xi) O
(h3) Rearranging h2ƒ"(xi) f(xi) 2f(xi-1)
f(xi-2) O (h3)
Second backward difference
16
Example of 2nd Backward FDD
Using data below calculate ƒ"(x2)
i 0 1 2 3 4 xi 1.0 3.0 5.0 7.0 9.0
ƒ(xi) 2.3 4.1 5.5 5.7 5.9 Second Backward
Finite-Divided-Difference at x2
f " (5.0) ? - 0.1 O (h)
17
Other Forms of Numerical Differentiation
What points are used for each form? Backward
, ƒ(xi-2), ƒ(xi-1), ƒ(xi),
ƒ(xi1), ƒ(xi2), Forward
, ƒ(xi-2), ƒ(xi-1), ƒ(xi), ƒ(xi1), ƒ(xi2),
Centered , ƒ(xi-2),
ƒ(xi-1), ƒ(xi), ƒ(xi1), ƒ(xi2),
18
Taylor Series and Truncation errors

• - Higher order divided differences.

- Second finite central divided difference
19
Other Forms of Numerical Differentiation
Forward
Centered
20
Taylor Series and Truncation errors

• Use of the Taylor series to calculate
  derivatives.
• First derivative with forward difference.

21
Taylor Series and Truncation errors

• First derivative with central differences.

22
Taylor Series and Truncation errors
Questions Which is a better approximation?
Forward, Centered, or Backward? Why? When
would you use which? Note We also can get
higher order forward, centered, and backward
difference derivative approximations CC
Chapter 23, tabulated in Figs. 23.1-3
23
Example Combining Roundoff and Truncation Error
Determine h to minimize the total error of a
forward finite-divided difference approximation
for

• Truncation Error

xi ? ? ? xi1

• Round-off Error

with e machine epsilon.
As a result
24
Example Combining Roundoff and Truncation Error
? Total error ? ?truncation error ? ?roundoff
error ?
E Total Error ?

NOTE Truncation error decreases as h decreases
Round-off error increases as h decreases
25
Example Combining Roundoff and Truncation Error
26
Example Combining Roundoff and Truncation Error
To minimize total error E with respect to h, set
the first derivative to zero
Solve for h and approximate f "(?) as f "(xi)
27
Example Combining Roundoff and Truncation Error
Linear Application Determine h that will
minimize total error for calculating f(x) for
f(x) ? x
at x 1
Using the first forward-divided-difference
approximation with error O(h) and a
5-decimal-digit machine e b1-t 101-5
10-4 0.0001 f '(x) ? f "(x) 0
28
Example Combining Roundoff and Truncation Error
f(xh) exact 3.1415 h ? (xh)
f(xh)-f(x) f(xh)-f(x)/h 0 3.1415 0.000001
3.1415 0 0 0.00001 3.1416 0.0001 10 0.0001 3.1
419 0.0004 4.0 0.001 3.1447 0.0032 3.2 0.01 3.
1730 0.0315 3.15 0.1 3. 4557 0.3142 3.142 1 6.
2831 3.1416 3.146
Underlined digits are subject to round-off error.
They are likely to be in error by one or two
units. This does not cause much problem when h
1, but causes large errors in the final result
when h lt 10-4.
29
Example Combining Roundoff and Truncation Error
Nonlinear Application Determine h for
minimizing the total error for computing f(x)
for ƒ(x) ex at x 3 Using the first
forward-divided-difference approximation with
error O(h) and a 5-decimal-digit machine
e b1-t 101-5 10-4 0.0001 f(x) f
'(x) f "(x) ex 20.0855
or about 0.01
30
Example Combining Roundoff and Truncation Error
full precision h f(xh)exh f(xh)-f(x) f(
xh)-f(x) f(xh)-f(x) h h 0 20.085 exact
20.085 0.00001 20.085 0.0 0 20.086 0.0001
20.087 0.002 20 20.086 0.001 20.105 0.020 20 20.0
96 0.01 20.287 0.202 20.2 20.18 0.1 22.198 2.113
21.13 21.12 1 54.598 34.513 34.523 34.512 Rou
ndoff Truncation
Underlined digits subject to roundoff error. Bold
digits in error due to truncation.