CSE 541 Numerical Methods

1
CSE 541Numerical Methods

• Machine Representation of Numbers

2
Condition of a Problem

• For a problem with input (data) x and output y,
• y f(x), the problem is said to be
  well-conditioned if small changes in x, lead to
  small changes in y
• Otherwise, we say the problem is ill-conditioned

3
Stability of an Algorithm

• An algorithm is said to be stable if small
  changes in the input x lead to small changes in
  the output y
• Otherwise, the algorithm is said to be unstable
• Stability indicates the sensitivity of an
  algorithm for solving a problem

4
Condition vs Stability

• Condition corresponds to . . . data
• Stability corresponds to . . . an algorithm
• Conclusions
• An ill-conditioned problem
• Very hard to get a good result with even the best
  algorithm
• A stable algorithm
• Given good data (not ill-conditioned), the
  algorithm will not yield drastically different
  results if round-off or small noise is added to
  the input data

5
Precision

• x .256834105
• Digit 2 is the most significant digit
• Digit 4 is the least significant digit
• Question What is the difference between Accuracy
  vs Precision?
• Accuracy How close your solution is to the
  actual solution. E.g., Missed the bulls-eye by 2
  inches.
• Precision How good your estimate of the
  accuracy, 2.00.003

6
Precision

• If you miss the bulls-eye, but repeatedly hit the
  same location, then it is precise, but inaccurate
• For instance, using Taylors series for the
  approximation of sin(x) (Maclaurin series), I can
  obtain a precise number for values of x larger
  than 2, but the solution would be inaccurate
• In a computer, precision is usually how many
  digits of accuracy your machine representation
  has (number of bits in the mantissa).

7
Integers

• Easy to represent on the computer and easy for
  the computer to perform integer arithmetic
  operations
• 00000000000000000000000000000000 ? 0
• 00000000000000000000000000000001 ? 1
• Range depends upon the machines word size
• Machines range
• 2n-1-1, where n is the number of bits allocated
  for an integer
• Use one bit for the sign
• See the /usr/include/limits.h file

8
Fixed-Point Numbers

• Assume you know a fixed range for your numbers
• (-999, 999)
• Say I need values for the surface temperatures of
  the earth
• Signed integers
• Restricted to only have three significant digits,
  no fractional values!!!
• How do we represent real numbers?
• Fixed-point numbers
• Keep the decimal point fixed
• For example, 8 significant digits
• Represent any temperature with the form xxx.xxxxx
  to allow for eight significant digits
• Numbers with a fixed range and number of
  significant digits

9
Fixed-point Arithmetic

• Easy to implement in most cases
• Ignore the decimal point for all operations
• Use integer calculations and then put the decimal
  point back in when needed (e.g. I/O)
• 31.45330 12.50000 ? 3145330 1250000 ?
  4395330 ? 43.95330

10
Floating-point numbers

• Allows for arbitrary precision around the decimal
  point
• A real number is represented by three parts
• Sign
• Mantissa 6024
• Exponent 20
• Represent the components as integers

6.0241023
11
Floating-Point Numbers

• CPU architects have choices on how many bits to
  allocate to each component
• Sign 1-bit
• Exponent n-bits
• Determines the range of numbers
• Mantissa whatever is left
• Determines the precision of the numbers

12
IEEE Floating Point

• Single-precision (typically 32-bits)
• Sign 1-bit (s)
• Exponent 8-bits (e)
• 0 lt e lt 255
• e 0, 0 and 0 are represented differently
• e 255, ? and -? (NaNs) have special
  representations
• Mantissa 23-bits (m)
• The mantissa is in the normalized form, meaning
  the first digit is a one. As such, it is not
  stored and we have 24-bits of precision.
• .5 lt m lt 1
• Notes
• e 127 ranges in -126, 127
• Smallest number is 2-126 is approximately 1.2 x
  10-38
• Largest number is 2128 is approximately 3.4 x
  1038
• Machine epsilon 1 e ¹ 1, 2-23 is approximately
  1.2 x 10-7, or about 6 decimal places

13
Limits for Floats

• Look at the C include file float.h
• define DBL_DIG 15
  / of decimal digits of precision /
• define DBL_EPSILON 2.2204460492503131e-016
  / smallest such that 1.0DBL_EPSILON ! 1.0 /
• define DBL_MANT_DIG 53
  / of bits in mantissa /
• define DBL_MAX 1.7976931348623158e308
  / max value /
• define DBL_MAX_10_EXP 308
  / max decimal exponent /
• define DBL_MAX_EXP 1024
  / max binary exponent /
• define DBL_MIN 2.2250738585072014e-308
  / min positive value /
• define DBL_MIN_10_EXP (-307)
  / min decimal exponent /
• define DBL_MIN_EXP (-1021)
  / min binary exponent /
• define _DBL_RADIX 2
  / exponent radix /
• define _DBL_ROUNDS 1
  / addition rounding near /
• define FLT_DIG 6
  / of decimal digits of precision /
• define FLT_EPSILON 1.192092896e-07F
  / smallest such that 1.0FLT_EPSILON ! 1.0 /
• define FLT_GUARD 0
• define FLT_MANT_DIG 24
  / of bits in mantissa /
• define FLT_MAX 3.402823466e38F
  / max value /

14
Loss of Significance

• Consider the error for x - y using 5 decimal
  digits of precision
• x .3721448693
• y . 3720214371
• x .37214
• y . 37202
• x-y .00012
• x-y .0001234322

15
Loss of Significance

• The relative error is
• However, the relative error of x and y is only
  1.310-5.
• We lost 3 significant digits!

16
Loss of Precision Theorem

• Precision loss for x - y
• Let x and y be normalized floating-point numbers,
  with x gt y gt 0. If
• for some positive integers p and q, then at most
  p and at least q significant binary bits are lost
  in the subtraction.
• Rule The closer two numbers, the greater the
  loss of significance.

17
Avoiding loss of Precision

• Use double precision or higher
• Modify the calculations to remove subtraction of
  numbers close together
• Consider as x approaches 0
• Reorder to remove the subtraction

18
How do they do that?

• Microsoft Windows ships with a simple calculator
  for its 32-bit operating system
• What would you expect, if you typed in 296?
• 296 79,228,162,514,264,337,593,543,950,336
• or

19
Performance vs. Accuracy

• With a 2.2GHz computer, you can do a lot of
  number crunching (two ops per clock cycle)
• Real-time applications
• Audio must be processed at 4KHz
• One trillion time steps of crash simulation
• Decide whether accuracy or performance is more
  important
• User interactivity (e.g., calculator)
• No need to calculate things faster than the
  users fat fingers can type them.
• Use slower, but more accurate algorithms

20
Divide by zero

• What would happen when you run this program?
• float x 0.230134789
• float y 0.230134788
• float inverse
• inverse 1.0 / (x - y)
• return 0

21
Resolve and Real Numbers

• One of the beautiful benefits of C is the
  ability to redefine or overload functions in a
  class hierarchy.
• With the CSE departments Resolve language, you
  used the type, Real.
• This class (as well as Resolve) is used to ensure
  that you do not divide by zero.
• Therefore, it overloads the operator/ method,
  performs a quick (???) check and then performs
  the divide (if safe).

22
Designing An Extended Precision Class

• C (and other object-oriented languages like
  Java), allow most of the basic language to be
  overloaded
• You can use this feature to define new arithmetic
  for
• limiting numbers 0.0 -gt 1.0, 32-63,
• embed debugging information print if valuepi
• Keep track of the error associated with the
  operation. This is called Interval Analysis
• Lets look at designing an extended precision
  class

23
Extended Precision with C

• Lets say we want over 1000-bits of precision for
  our application
• Furthermore, lets assume our numbers encode
  temperatures and only range from -273 to 65,000
  C.
• Key questions
• What should our basic number format be?
• What operations are we supporting?
• How do we implement those operations?

24
Decimal Type

• C has a new decimal type
• 128-bit format
• 28 significant figures
• That should handle Bill Gates check-book!!!
• Fixed-point, floating point or integer?
• Bit layout?

25
Other C features

• Keywords checked or unchecked
• Controls whether to throw an OverflowException or
  not
• For integer arithmetic only
• int wontThrow unchecked( System.Int32.MaxValue
  1)
• int willThrow checked( System.Int32.MaxValue
  1)
• Can also be used in blocks or as a compiler
  option
• Many exceptions can be thrown and caught