KrylovSubspace Methods II

1
Krylov-Subspace Methods - II
Lecture 7 Alessandra Nardi
Thanks to Prof. Jacob White, Deepak Ramaswamy,
Michal Rewienski, and Karen Veroy
2
Last lectures review

• Overview of Iterative Methods to solve Mxb
• Stationary
• Non Stationary
• QR factorization
• Modified Gram-Schmidt Algorithm
• Minimization View of QR
• General Subspace Minimization Algorithm
• Generalized Conjugate Residual Algorithm
• Krylov-subspace
• Simplification in the symmetric case
• Convergence properties
• Eigenvalue and Eigenvector Review
• Norms and Spectral Radius
• Spectral Mapping Theorem

3
Arbitrary Subspace MethodsResidual Minimization
4
Arbitrary Subspace MethodsResidual Minimization
Use Gram-Schmidt on Mwis!
5
Krylov Subspace Methods
Krylov Subspace
kth order polynomial
6
Krylov Subspace MethodsSubspace Generation
The set of residuals also can be used as a
representation of the Krylov-Subspace
Generalized Conjugate Residual Algorithm Nice
because the residuals generate next search
directions
7
Krylov-Subspace MethodsGeneralized Conjugate
Residual Method (k-th step)
Determine optimal stepsize in kth search direction
Update the solution (trying to minimize residual)
and the residual
Compute the new orthogonalized search direction
(by using the most recent residual)
8
Krylov-Subspace MethodsGeneralized Conjugate
Residual Method (Computational Complexity for
k-th step)
Vector inner products, O(n) Matrix-vector
product, O(n) if sparse
Vector Adds, O(n)
O(k) inner products, total cost O(nk)
If M is sparse, as k ( of iters) approaches n,
Better Converge Fast!
9
Summary

• What is an iterative non stationary method
  x(k1) x(k)akpk
• How search to calculate
• Search directions (pk)
• Step along search directions (ak)
• Krylov Subspace ? GCR
• GCR is O(k2n)
• Better converge fast!
• ?? Now look at convergence properties of GCR

10
Krylov Methods Convergence AnalysisBasic
properties
11
Krylov Methods Convergence AnalysisOptimality of
GCR poly
GCR Optimality Property
Therefore
Any polynomial which satisfies the constraints
can be used to get an upper bound on
12
Eigenvalues and eigenvectors reviewInduced norms
Theorem Any induced norm is a bound on the
spectral radius
Proof
13
Useful Eigenproperties Spectral Mapping Theorem
Given a polynomial
Apply the polynomial to a matrix
Then
14
Krylov Methods Convergence AnalysisOverview
Matrix norm property
GCR optimality property
where is any (k1)-th order
polynomial subject to ? may be
used to get an upper bound on
15
Krylov Methods Convergence AnalysisOverview

• Review on eigenvalues and eigenvectors
• Induced norms relate matrix eigenvalues to the
  matrix norms
• Spectral mapping theorem relate matrix
  eigenvalues to matrix polynomials
• Now ready to relate the convergence properties of
  Krylov Subspace methods to eigenvalues of M

16
Krylov Methods Convergence AnalysisNorm of
matrix polynomials
Cond(V)
17
Krylov Methods Convergence AnalysisNorm of
matrix polynomials
18
Krylov Methods Convergence AnalysisImportant
observations
1) The GCR Algorithm converges to the exact
solution in at most n steps
2) If M has only q distinct eigenvalues, the GCR
Algorithm converges in at most q steps
19
Krylov Methods Convergence AnalysisConvergence
for MTM - Residual Polynomial
If M MT then
1) M has orthonormal eigenvectors
2) M has real eigenvalues
20
Krylov Methods Convergence AnalysisResidual
Polynomial Picture (n10)
1
evals(M)
- 5th order poly
- 8th order poly
21
Krylov Methods Convergence AnalysisResidual
Polynomial Picture (n10)
Strategically place zeros of the poly
22
Krylov Methods Convergence AnalysisConvergence
for MTM Polynomial min-max problem
23
Krylov Methods Convergence AnalysisConvergence
for MTM Chebyshev solves min-max
The Chebyshev Polynomial

24
Chebychev Polynomials minimizing over 1,10
25
Krylov Methods Convergence AnalysisConvergence
for MTM Chebyshev bounds
26
Krylov Methods Convergence AnalysisConvergence
for MTM Chebyshev result
27
Krylov Methods Convergence AnalysisExamples
For which problem will GCR Converge Faster?
28
Which Convergence Curve is GCR?
Iteration
29
Krylov Methods Convergence AnalysisChebyshev is
a bound
GCR Algorithm can eliminate outlying eigenvalues
by placing polynomial zeros directly on them.
30
Iterative Methods - CG
Why ? How?

• Convergence is related to
• Number of distinct eigenvalues
• Ratio between max and min eigenvalue

31
Summary

• Reminder about GCR
• Residual minimizing solution
• Krylov Subspace
• Polynomial Connection
• Review Eigenvalues
• Induced Norms bound Spectral Radius
• Spectral mapping theorem
• Estimating Convergence Rate
• Chebyshev Polynomials