MATH 685 CSI 700 OR 682 Lecture Notes

1
MATH 685/ CSI 700/ OR 682 Lecture Notes

• Lecture 6.
• Eigenvalue problems.

2
Eigenvalue problems

• Eigenvalue problems occur in many areas of
  science and engineering, such as structural
  analysis
• Eigenvalues are also important in analyzing
  numerical
• methods
• Theory and algorithms apply to complex matrices
  as well
• as real matrices
• With complex matrices, we use conjugate
  transpose, AH, instead of usual transpose, AT

3
Formulation
Matrix expands or shrinks any vector lying in
direction of eigenvector by scalar
factor Expansion or contraction factor is given
by corresponding eigenvalue Eigenvalues and
eigenvectors decompose complicated behavior of
general linear transformation into simpler actions
4
Examples
5
Characteristic polynomial
6
Example
7
Companion matrix
8
Characteristic polynomial

• Computing eigenvalues using characteristic
  polynomial is not recommended because of
• work in computing coefficients of characteristic
  polynomial
• sensitivity of coefficients of characteristic
  polynomial
• work in solving for roots of characteristic
  polynomial
• Characteristic polynomial is powerful theoretical
  tool but usually not useful computationally

9
Example
10
Diagonalizability
11
Eigenspaces
12
Some relevant definitions
13
Examples
14
Examples
15
Properties of eigenvalue problems

• Properties of eigenvalue problem affecting choice
  of algorithm and software
• Are all eigenvalues needed, or only a few?
• Are only eigenvalues needed, or are corresponding
  eigenvectors also needed?
• Is matrix real or complex?
• Is matrix relatively small and dense, or large
  and sparse?
• Does matrix have any special properties, such as
  symmetry, or is it general matrix?
• Condition of eigenvalue problem is sensitivity of
  eigenvalues and eigenvectors to changes in matrix
• Conditioning of eigenvalue problem is not same as
  conditioning of solution to linear system for
  same matrix
• Different eigenvalues and eigenvectors are not
  necessarily equally sensitive to perturbations in
  matrix

16
Conditioning of eigenvalues
17
Conditioning of eigenvalues
18
Problem transformations
19
Similarity transformation
20
Similarity transformation
21
Diagonal form

• Eigenvalues of diagonal matrix are diagonal
  entries, and eigenvectors are columns of identity
  matrix
• Diagonal form is desirable in simplifying
  eigenvalue problems for general matrices by
  similarity transformations
• But not all matrices are diagonalizable by
  similarity transformation
• Closest one can get, in general, is Jordan form,
  which is nearly diagonal but may have some
  nonzero entries on first superdiagonal,
  corresponding to one or more multiple eigenvalues

22
Triangular form
23
Block triangular form
24
Forms attainable by similarity
25
Power iteration
26
Convergence of Power iteration
27
Example
28
Limitations
29
Normalized Power iteration
30
Example
31
Geometric interpretation
32
Power Iteration with Shift
In earlier example, for instance, if we pick
shift of s 1, (which is equal to other
eigenvalue) then ratio becomes zero and method
converges in one iteration In general, we would
not be able to make such fortuitous choice, but
shifts can still be extremely useful in some
contexts, as we will see later
33
Inverse iteration
Inverse iteration converges to eigenvector
corresponding to smallest eigenvalue of A.
Eigenvalue obtained is dominant eigenvalue of
A-1, and hence its reciprocal is smallest
eigenvalue of A in modulus
34
Example
35
Shifted inverse iteration
36
Rayleigh Quotient
37
Example
38
Rayleigh Quotient iteration
39
Rayleigh Quotient iteration
40
Deflation
41
Deflation
42
Deflation
43
Simultaneous Iteration
44
Orthogonal iteration
45
QR iteration
46
QR iteration
47
Example
48
QR iteration with shifts
49
Example
50
Preliminary reduction
51
Preliminary reduction
52
Preliminary reduction
53
Cost of QR iteration
54
Krylov subspaces methods
55
Krylov subspaces methods
56
Arnoldi iteration
57
Arnoldi iteration
58
Arnoldi iteration
59
Lanczos Iteration
60
Lanczos iteration
61
Lanczos iteration
62
Krylov subspace methods cont.
63
Example of Lanczos iteration
64
Jacobi method
65
Jacobi method
66
Plane rotation
67
Jacobi method cont.
68
Jacobi method cont.
69
Jacobi method example
70
(No Transcript)
71
Process continues until off-diagonal entries
reduced to as small as desired Result is
diagonal matrix orthogonally similar to original
matrix, with the orthogonal similarity
transformation given by product of plane rotations
72
Other methods (spectrum-slicing)
73
Sturm sequence
74
Divide-and-conquer algorithm
75
Relatively robust representation
76
Generalized eigenvalue problems
77
QZ algorithm
78
Computing SVD