Backward Thinking

1
Backward Thinking

• Confessions of a Numerical Analyst
• Keith Evan Schubert

2
Simple Problem

• Consider the problem axb
• The resulting x value is

3
Simple Problem 2

• Consider the problem axb
• The resulting x value is

4
Whats Up?

• The condition number (sensitivity to
  perturbations) is about 400.
• A condition number of 1 is perfect.
• Perturbation is 0.01, so 0.014004.
• Components of x can vary by this much!

5
What Can We Do?

• Rather than solve it the standard way
• Xa\b
• X(ATA)-1atb
• Consider the following
• X(ATA?i)-1atb
• ? .01
• Then

6
Lucky Guess?
7
Does It Always Work?

• No
• Consider ???
• X?0
• Consider ??si2 (si is singular value of A)
• X??
• Picking the wrong value can get junk

8
Skyline

• Consider a 1 dimensional picture
• Use height instead of color
• Result looks like the silhouette of a citys
  skyline
• Have smog which blurs and softens
• Dont know exactly how much blur
• Want to get sharp edges

9
Getting Garbage
10
Getting Improvement
11
Why Backward?

• Forward errors
• Explicitly account for each error source
• (Xd1)(yd2)xy(yd1xd2d1d2)
• Backward errors
• Check that my algorithm acting on data will give
  me a solution that is near to the actual system
  acting on a nearby set of data
• I.E. My algorithm with good data should do about
  as well as a perfect calculation on ok data

12
Picture Please!
Inherent errors in A
b
Perfect Calculations
b
Errors due to algorithm
best
My Algorithm
Actual Data (x)
Nearby Data (x)
13
Least Squares

• Usually we dont have an invertible matrix
• Need to find an estimated solution
• Criterion minimize ax-b
• Normal equation
• ATA x ATb
• Solution
• X (ATA)-1atb

14
Backward Error

• Criterion minimize Ax-b/(A xb)
• Normal Equations
• Solution

15
Non Convex
16
Finding The Root
17
Look At Critical Region
18
Informal Algorithm

• Get (A,b)
• svd(A) ? u1 u2,?,v
• U1b ? b1
• Use rootfinder (bisection, Newton, etc.) to get ?
  in -sn2,0
• vT(?2- ?I)-1 ? b1 ? x

19
What You Get
20
Least Squares
21
Total Least Squares
22
Tikhonov
23
Backward Error
24
Original
25
Comparison
26
Final Thoughts

• BE is always optimistic in that it presumes that
  the real system is better
• Even with this it is robust
• There is a perturbed version of this algorithm
  which can be either optimistic or pessimistic
• That version is not fully proven