A%20History%20of%20Numerical%20Analysis%20Ideas

1
A History of Numerical Analysis Ideas
Prepared for CS 378 History of Computing October
14, 2003

• Alan Kaylor Cline
• Department of Computer Sciences
• The University of Texas at Austin

2
What is Different in Numerical Computing?
3
What is Different in Numerical Computing?
Well, its numbers
4
Scientific Computing vs.Numerical Analysis
5
Your Original CS Department
1966
6
Your Original CS Department
1966
10 faculty with 2 numerical analysts
7
Is there anything special with numerical
computing?
8
A Small Example
A computation of p
9
Simple iteration
10
2 2.828427124746190 3 3.061467458920719 4
3.121445152258053 5 3.136548490545941 6
3.140331156954739 7 3.141277250932757 8
3.141513801144145 9 3.141572940367883 10
3.141587725279961 11 3.141591421504635 12
3.141592345611077 13 3.141592576545004 14
3.141592633463248 15 3.141592654807589 16
3.141592645321215 17 3.141592607375720 18
3.141592910939673 19 3.141594125195191 20
3.141596553704820 21 3.141596553704820 22
3.141674265021758 23 3.141829681889202 24
3.142451272494134 25 3.142451272494134 26
3.162277660168380 27 3.162277660168380 28
3.464101615137754 29 4.000000000000000 30
0.000000000000000 31 0.000000000000000
Result of 15 digit computation
Red digits are correct
White and green digits are incorrect
11
2 2.828427124746190 3 3.061467458920719 4
3.121445152258053 5 3.136548490545941 6
3.140331156954739 7 3.141277250932757 8
3.141513801144145 9 3.141572940367883 10
3.141587725279961 11 3.141591421504635 12
3.141592345611077 13 3.141592576545004 14
3.141592633463248 15 3.141592654807589 16
3.141592645321215 17 3.141592607375720 18
3.141592910939673 19 3.141594125195191 20
3.141596553704820 21 3.141596553704820 22
3.141674265021758 23 3.141829681889202 24
3.142451272494134 25 3.142451272494134 26
3.162277660168380 27 3.162277660168380 28
3.464101615137754 29 4.000000000000000 30
0.000000000000000 31 0.000000000000000 . . .
Result of 15 digit computation
Red digits are correct
White and green digits are incorrect
p 0 ?
12
Wheres the problem?
13
Wheres the problem?
is calculated as zero
14
Lets replace
with the algebraically identical expression
15
New iteration
results in
16
2 2.828427124746190 3 3.061467458920719 4
3.121445152258053 5 3.136548490545940 6
3.140331156954753 7 3.141277250932773 8
3.141513801144301 9 3.141572940367091 10
3.141587725277160 11 3.141591421511200 12
3.141592345570118 13 3.141592576584872 14
3.141592634338563 15 3.141592648776985 16
3.141592652386591 17 3.141592653288992 18
3.141592653514593 19 3.141592653570993 20
3.141592653585093 21 3.141592653588618 22
3.141592653589499 23 3.141592653589719 24
3.141592653589774 25 3.141592653589788 26
3.141592653589792 27 3.141592653589793 28
3.141592653589793 29 3.141592653589793 30
3.141592653589793 31 3.141592653589793
2 2.828427124746190 3 3.061467458920719 4
3.121445152258053 5 3.136548490545941 6
3.140331156954739 7 3.141277250932757 8
3.141513801144145 9 3.141572940367883 10
3.141587725279961 11 3.141591421504635 12
3.141592345611077 13 3.141592576545004 14
3.141592633463248 15 3.141592654807589 16
3.141592645321215 17 3.141592607375720 18
3.141592910939673 19 3.141594125195191 20
3.141596553704820 21 3.141596553704820 22
3.141674265021758 23 3.141829681889202 24
3.142451272494134 25 3.142451272494134 26
3.162277660168380 27 3.162277660168380 28
3.464101615137754 29 4.000000000000000 30
0.000000000000000 31 0.000000000000000
p correct to all digits
17
Boring Is that all there is to numerical
analysis?
18
Not so boring if the result of this computation
affects
19
Not so boring if the result of this computation
affects

• The ability of the next plane you fly to stay in
  the air

20
Not so boring if the result of this computation
affects

• The ability of the next plane you fly to stay in
  the air
• The integrity of the next bridge you cross

21
Not so boring if the result of this computation
affects

• The ability of the next plane you fly to stay in
  the air
• The integrity of the next bridge you cross
• The state of the economy on which you live

22
Not so boring if the result of this computation
affects

• The ability of the next plane you fly to stay in
  the air
• The integrity of the next bridge you cross
• The state of the economy on which you live
• The path of a missile that isnt intended to
  strike you

23
So what are the common problems of numerical
analysis?
24
So what are the common problems of numerical
analysis?
Application areas
25
So what are the common problems of numerical
analysis?
Application areas

• Petroleum modeling

26
So what are the common problems of numerical
analysis?
Application areas

• Petroleum modeling
• Atomic energy including weapons

27
So what are the common problems of numerical
analysis?
Application areas

• Petroleum modeling
• Atomic energy including weapons
• Weather modeling

28
So what are the common problems of numerical
analysis?
Application areas

• Petroleum modeling
• Atomic energy including weapons
• Weather modeling
• Other modeling such as aircraft and automobile

29
So what are the common problems of numerical
analysis?
Algorithm areas
30
So what are the common problems of numerical
analysis?
Algorithm areas

• Linear Equations

31
So what are the common problems of numerical
analysis?
Algorithm areas

• Linear Equations
• Nonlinear equations - single and systems

32
So what are the common problems of numerical
analysis?
Algorithm areas

• Linear Equations
• Nonlinear equations - single and systems
• Optimization

33
So what are the common problems of numerical
analysis?
Algorithm areas

• Linear Equations
• Nonlinear equations - single and systems
• Optimization
• Data Fitting - interpolation and approximation

34
So what are the common problems of numerical
analysis?
Algorithm areas

• Linear Equations
• Nonlinear equations - single and systems
• Optimization
• Data Fitting - interpolation and approximation
• Integration

35
So what are the common problems of numerical
analysis?
Algorithm areas

• Linear Equations
• Nonlinear equations - single and systems
• Optimization
• Data Fitting - interpolation and approximation
• Integration
• Differential Equations - ordinary and partial

36
Didnt we study that stuff in math classes?
37
Didnt we study that stuff in math classes?
Yes, but as the Pi Example shows, math classes
are just the beginning
38
Lets get back to history
39
Why were computers used primarily for numerical
problems initially?
40

• Why were computers used primarily for numerical
  problems initially?
• Mathematicians and engineers designed them

41

• Why were computers used primarily for numerical
  problems initially?
• Mathematicians and engineers designed them
• A history of algorithms in that area

42

• Why were computers used primarily for numerical
  problems initially?
• Mathematicians and engineers designed them
• A history of algorithms in that area
• Immediate war-time and post-war-time
  applications

43

• Why were computers used primarily for numerical
  problems initially?
• Mathematicians and engineers designed them
• A history of algorithms in that area
• Immediate war-time and post-war-time
  applications
• Applications did not depend upon having a large
  number of computers

44

• Why were computers used primarily for numerical
  problems initially?
• Mathematicians and engineers designed them
• A history of algorithms in that area
• Immediate war-time and post-war-time
  applications
• Applications did not depend upon having a large
  number of computers
• However, there were non-numerical examples
  ENIGMA

45
What were the major computing ideas that arose in
numerical analysis?
46
What were the major computing ideas that arose in
numerical analysis?

• Backward error analysis

47
What were the major computing ideas that arose in
numerical analysis?

• Backward error analysis

input
output
48
What were the major computing ideas that arose in
numerical analysis?

• Backward error analysis

true operation
input
output
49
What were the major computing ideas that arose in
numerical analysis?

• Backward error analysis

true operation
approximate operation
input
output
50
What were the major computing ideas that arose in
numerical analysis?

• Backward error analysis

true operation
error
approximate operation
input
output
51
What were the major computing ideas that arose in
numerical analysis?

• Backward error analysis

true operation
approximate operation
input
output
52
What were the major computing ideas that arose in
numerical analysis?

• Backward error analysis

true operation
backward error
approximate operation
input
output
53
What were the major computing ideas that arose in
numerical analysis?

• Backward error analysis
• FORTRAN

54
What were the major computing ideas that arose in
numerical analysis?

• Backward error analysis
• FORTRAN

• Mathematical based
• Computationally Efficient
• Portable
• Standardized 3 times

55
What were the major computing ideas that arose in
numerical analysis?

• Backward error analysis
• FORTRAN
• Mathematical software packages

56
What were the major computing ideas that arose in
numerical analysis?

• Backward error analysis
• FORTRAN
• Mathematical software packages

• IMSL
• Eispack
• Linpack

57
What were the major computing ideas that arose in
numerical analysis?

• Backward error analysis
• FORTRAN
• Mathematical software packages
• NANET

58
What were the major computing ideas that arose in
numerical analysis?

• Backward error analysis
• FORTRAN
• Mathematical software packages
• NANET

• Weekly information about people, problems, and
  papers
• Software repository

59
What were the major computing ideas that arose in
numerical analysis?

• Backward error analysis
• FORTRAN
• Mathematical software packages
• NANET
• Matlab, Mathematica

60
What were the major computing ideas that arose in
numerical analysis?

• Backward error analysis
• FORTRAN
• Mathematical software packages
• NANET
• Matlab, Mathematica

Scientific computing environments
61
What were the major computing ideas that arose in
numerical analysis?

• Backward error analysis
• FORTRAN
• Mathematical software packages
• NANET
• Matlab, Mathematica
• Super computers - Parallelism