Chapter 4 Eigenvalues and Eigenvectors

1
Chapter 4 Eigenvalues and Eigenvectors

• 4.1 Introduction to Eigenvalues and Eigenvectors
• 4.2 Determinats
• 4.3 Eigenvalues and Eigenvectors of n n
  Matrices
• 4.4 Similarity and Diagonalization
• 4.5 Iterative Methods for Computing Eigenvalues
• 4.6 Applications and the Perron-Frobenius Theorem

2
4.1 Introduction to Eigenvalues and Eigenvectors
Definition Let A be an n n matrix. A scalar ?
is called an eigenvalue of A if there is a
nonzero vector x such that Ax ?x. Such a
vector x is called an eigenvector of A
corresponding to ?.
3
Definition Let A be an n n matrix and let ? be
an eigenvalue of A. The collection of
all eigenvectors corresponding to ?, together
with the zero vector, is called the eigenspace of
? and is denoted by E?.
4
(No Transcript)
5
Another way to think of eigenvectors
geometrically is to draw x and Ax
head-to-tail. Then x will be an eigenvector of A
if and only if x and Ax are aligned in a
straight line. In figure, x is an eigenvector of
A but y is not.
y
Ay
4
3
y
2
Ax
1
x
x
1
2
3
4
? is an eigenvalue of A if and only if the null
space of A ?I is nontrivial. The det (A ?I )
0
6
4.2 Determinants
7
(No Transcript)
8
Another method for calculating the determinant of
a 3 3 matrix is analogous to the method for
calculating the determinant of a 2 2 matrix.
This method gives a11a22a33 a12a23a31
a13a21a32 a31a22a13 a32a23a11 a33a21a12
9
Determinants of n n Matrices
10
(No Transcript)
11
Since Cij (-1)i j det Aij, each cofactor is
plus or minus the corresponding minor, with the
correct sign given by the term (-1)i j.
12
(No Transcript)
13
Theorem 4.2 The determinant of a triangular
matrix is the product of the entries on its
main diagonal. Specifically. if A aij is an n
n triangular matrix then det A a11a22 ann

• Properties of Determinants

Theorem 4.3 Let A aij be a square
matrix. a. If A has a zero row (column), then
det A 0. b. If B is obtained by interchanging
two rows (columns) of A, then det B - det A. c.
If A has two identical rows (columns), then det A
0. d. If B is obtained by multiplying a row
(column) of A by k, then det B k det A. e. If
A, B, and C are identical except that the ith row
(column) of C is the sum of the ith rows
(columns) of A and B, then det C det A det
B. f. If B is obtained by adding a multiple of
one row (column) of A to another row (column),
then det B det A.
14
Determinants of Elementary Matrices
Theorem 4.4 Let E be an n n elementary
matrix. a. If E results from interchanging two
rows of In, then det E -1. b. If E results from
multiplying one row of In by k, then det E
k. c. If E results from adding a multiple of one
row of In to another row, then det E 1.
Lemma 4.5 Let B be an n n matrix and let E be
an n n elementary matrix. Then det(EB)
(det E)(det B)
Theorem 4.6 A square matrix A is invertible if
and only if det A ? 0.
15
Determinants and Matrix Operations
Theorem 4.7 If A is an n n matrix, then
det(kA) kn det A
Theorem 4.8 If A and B are n n matrices, then
det(AB) (det A)(det B)
16
Example Applying Theorem 4.8 to
and , we find that
and that det A 4, det B 3, and
det(AB) 12 4.3 (det A)(det B)
17
Sol
Theorem 4.10 For any square matrix A, det A
det AT
18
Cramers Rule and the Adjoint
For an n n matrix A and a vector b in Rn, let
Ai(b) denote the matrix obtained by replacing the
ith column of A by b. That is,
19
(No Transcript)
20
However,
which is the (j, i)-cofactor of A xij (1/det A)
Cji, so A-1 X (1/det A) Cji (1/det A)
CijT. The inverse of A is the transpose of the
matrix of cofactors of A, divided by
the determinant of A.
21
(No Transcript)
22
The Laplace Expansion Theorem
Lemma 4.13 Let A be an n n matrix.
Then a11C11 a12C12 a1nC1n det A
a11C11 a21C21 an1Cn1
Lemma 4.14 Let A be an n n matrix and let B
be obtained by interchanging any two
rows (columns) of A. Then det B -det A
23
4.3 Eigenvalues and Eigenvectors of n n
Matrices
The eigenvalues of a square matrix A are
precisely the solutions ? of the
equation det(A -?I) 0
When we expand det(A -?I), we get a polynomial in
?, called the characteristic polynomial of A. The
equation det(A -?I) 0 is called the
characteristic equation of A. If
,
24
Let A be an n n matrix. 1. Compute the
characteristic polynomial det(A -?I) of A. 2.
Find the eigenvalues of A by solving the
characteristic equation det(A -?I) 0 for
?. 3. For each eigenvalue ?, find the null space
of the matrix A ?I. This is the eigenspace
E?, the nonzero vectors of which are the
eigenvectors of A corresponding to ?. 4. Find
a basis for each eigenspace.
25
(No Transcript)
26
(No Transcript)
27
Define the algebraic multiplicity of an
eigenvalue to be its multiplicity as a root
of the characteristic equation. Define the
geometric multiplicity of an eigenvalue ? to be
dim E?, the dimension of its corresponding
eigenspace.
28
Theorem 4.15 The eigenvalues of a triangular
matrix are the entries on its main diagonal.
Theorem 4.16 A square matrix A is invertible if
and only if 0 is not an eigenvalue of A.
29
Theorem 4.17 The Fundamental Theorem of
Invertible Matrices Version 3 Let A be an n n
matrix. The following statements are
equivalent a. A is invertible. b. Ax b has
a unique solution for every b in Rn. c. Ax 0
has only the trivial solution. d. The reduced
row echelon form of A is In. e. A is a product
of elementary matrices. f. rank(A) n g.
nullity(A) 0 h. The column vectors of A are
linearly independent. i. The column vectors of A
span Rn. j. The column vectors of A form a basis
for Rn. k. The row vectors of A are linearly
independent. l. The row vectors of A span Rn. m.
The row vectors of A form a basis for Rn. n. det
A ? 0 o. 0 is not an eigenvalue of A.
30
Theorem 4.18 Let A be a square matrix with
eigenvalue ? and corresponding eigenvector x. a.
For any positive integer n, ?n is an eigenvalue
of An with corresponding eigenvector x. b.
If A is invertible, then 1/? is an eigenvalue of
A-1 with corresponding eigenvector x. c. For
any integer n, ?n is an eigenvalue of An with
corresponding eigenvector x.
31
(No Transcript)
32
Theorem 4.20 Let A be an n n matrix and let
?1, ?2, , ?m be distinct eigenvalues of A
with corresponding eigenvectors v1, v2, vm.
Then v1, v2, , vm are linear independent.
33
4.4 Similarity and Diagonalization

• Similar Matrices

Definition Let A and B be n n matrices. We say
that A is similar to B if there is an invertible
n n matrix P such that P-1AP B. If A is
similar to B, we write A B.
34
Theorem 4.21 Let A, B and C be n n
matrices. a. A A. b. If A B, then B A. c.
If A B and B C, then A C.
Theorem 4.22 Let A and B be n n matrices with
A B. Then a. det A det B b. A is invertible
if and only if B is invertible. c. A and B have
the same rank. d. A and B have the same
characteristic polynomial. e. A and B have the
same eigenvalues.
35
(No Transcript)
36
Diagonalization
Definition An n n matrix A is diagonalizable
if there is a diagonal matrix D such that A
is similar to Dthat is, if there is an
invertible n n matrix P such that P-1AP D.
Theorem 4.23 Let A be an n n matrix. Then A
is diagonalizable if and only if A has n
linearly independent eigenvectors. More
precisely, there exist an invertible matrix P and
a diagonal matrix D such that P-1AP D if and
only if the columns of P are n linearly
independent eigenvectors of A and the diagonal
entries of D are the eigenvalues of A
corresponding to the eigenvectors in P in the
same order.
37
Theorem 4.24 Let A be an n n matrix and let
?1, ?2, , ?k be distinct eigenvalues of A. If
Bi is a basis for theeigehspace E?i, then B B1
? B2 ? ? Bk (i.e., the total collection of
basis vectors for all of the eigenspaces) is
linearly independent.
Theorem 4.25 If A is an n n matrix with n
distinct eigenvalues, then A is diagonalizable.
38
Example The matrix
has eigenvalues ?1 2, ?2 5, and ?3 -1, by
Theorem 4.15.Since these are three distinct
eigenvalues for a 3 3 matrix, A is
diagonalizabble, by Theorem 4.25
Lemma 4.26 If A is an n n matrix, then the
geometric multiplicity of each eigenvalue is less
than or equal to its algebraic multiplicity.
39
Theorem 4.27 The Diagonalization Theorem Let
A be an n n matrix whose distinct eigenvalues
are ?1, ?2, , ?k. The following statements are
equivalent a. A is diagonalizable. b. The union
B of the bases of the eigenspaces of A (as in
Theorem 4.24) contains n vectors. c. The
algebraic multiplicity of each eigenvalue equals
its geometric multiplicity.
40
(No Transcript)
41
(No Transcript)
42
(No Transcript)
43
4.5 Iterative Methods for Computing Eigenvalues

• The Power Method

Theorem 4.28 Let A be an n n diagonalizable
matrix with dominant eigenvalue ?1. Then
there exists a nonzero vector x0 such that the
sequence of vectors xk defined by x1 Ax0, x2
Ax1, x3 Ax2, , xk Axk-1, approaches a
dominant eigenvector of A.
44
The Power Method
45
x3
x1
x2
x0
The ratio lk of the first component of xk 1 to
that of xk will approach ?1 as k increases.
46
There is a drawback to the method The components
of the iterates xk get very large very quickly
and can cause significant roundoff errors. To
avoid this drawback, we can multiply each iterate
by some scalar that reduces the magnitude of its
components. An easier method is to divide each
xk by the component with the maximum
absolute value, so that the largest component is
now 1. This method is called scaling. Thus, if
mk denotes the component of xk with the maximum
absolute value, we will replace xk by yk
(1/mk)xk. This method is called the power method.
47
The Power Method
The Power Method Let A be a diagonalizable n n
matrix with a corresponding dominant eigenvalue
?1. 1. Let x0 y0 be any initial vector in Rn
whose largest component is 1. 2. Repeat the
following steps for k 1, 2, (a) Compute
xk Ayk 1 (b) Let mk be the component of
xk with the largest absolute value. (c) Set
yk (1/mk)xk. For most choices of x0, mk
converges to the dominant eigenvalue ?1 and
yk converges to a dominant eigenvector.
48
The Shifted Power Method and the Inverse Power
Method
The shifted power method uses the observation
that , if ? is an eigenvalue of A, then ? a is
an eigenvalue of A aI for any scalar a. Thus,
if ?1 is the dominant eigenvalue of A, the
eigenvalues of A ?1I will be 0, ?2 ?1, ?3
?1, , ?n ?1.
49
Therefore, ?2 ?1 -3, so ?2 ?1 3 2 3
-1 is the second eigenvalue of A.
50
If A is invertible with eigenvalue ?, then A-1
has eigenvalue 1/?. Therefore, if we apply the
power method to A-1, its dominant eigenvalue will
be the reciprocal of the smallest eigenvalue of
A. To use this inverse power method, we follow
the same steps as in the power method, xk A-1yk
1.
51
Thus, , so . Then
we get x2 from Ax2 y1
52
The Shifted Inverse Power Method
If a scalar a is given, the shifted inverse power
method will find the eigenvalue ? of A that is
closest to a.
53
(No Transcript)
54
(No Transcript)
55
Gerschgorins Theorem
56
Sol (a) The two Gerschgorin disks arecentered at
2 and -3 with radii 1 and 2. The
characteristic polynomial of A is ?2 ? 8
57
(b) The two Gerschgorin disks are centered at 1
and 3 with radii -3 3and 2. The
characteristic polynomial of A is ?2 4? 9
58
Theorem 4.29 Gerschorins Disk Theorem Let
A be an n n (real or complex) matrix. Then
every eigenvalue of A is contained Within a
Gerschgorin disk.
59
because xi ? 0. Taking absolute values and using
properties of absolute value , we obtain
because xj ? xi for j ? i.
This establishes that the eigenvalue ? is
contained within the Gerschgorin disk centered
at aii with radius ri.
60
Gerschgorins Theorem tells us that the
eigenvalues of A are contained with three disks
centered at 2, 6, and 8 with radii 1, 1, and 2.
61
Theorem 4.29 tells us that the same three
eigenvalues of A are contained in disks centered
at 2, 6, and 8with radii 5/2, 1 and 1/2
62
4.6 Applications and the Perron-Frobenius Theorem

• Markov Chains

Theorem 4.30 If P is the n n transition
matrix of a Markov chain, then 1 is an eigenvalue
of P.
Theorem 4.31 Let P be an n n transition
matrix with eigenvalue ?. a. ? ? 1 b. If P is
regular and ? ? 1, then ? lt 1.
63
So, taking , that
64
(No Transcript)
65
Lemma 4.32 Let P be a regular n n transition
matrix. If P is diagonalizable, then the
dominant eigenvalue ?1 1 has algebraic
multiplicity 1.
Theorem 4.33 Let P be a regular n n
transition matrix. Then as k ? 8, Pk approaches
an n n matrix L whose columns are identical,
each equal to the same vector x. This vector x is
a steady state probability vector for P.
Theorem 4.34 Let P be a regular n n
transition matrix, with x the steady state
probability vector for P, as in Theorem 4.33.
Then, for any initial probability vector x0, the
sequence of iterates xk approaches x.
66
Population Growth
67
(No Transcript)
68
Theorem 4.35 Every Leslie matrix has a unique
positive eigenvalue and a corresponding
eigenvector with positive components.

• The Perron-Frobenius Theorem

Theorem 4.36 Perrons Theorem Let A be a
positive n n matrix. Then A has a real
eigenvalue ?1 tith the following properties a.
?1 gt 0 b. ?1 has a corresponding positive
eigenector. c. If ?is any other eigenvalue of A,
then ? ? ?1
69
A square matrix A is called reducible if ,
subject to some permutation of the rows and the
same permutation of the columns. A square matrix
A that is not reducible is called
irreducible. If Ak gt O for some k, then A is
called primitive.
70
Linear Recurrence Relations
The Fibonacci numbers are the numbers in the
sequence where, after the first two terms, each
new term is obtained by summing the two terms
preceding it. fn fn 1 fn 2
Definition Let (xn) (x0, x1, x2, ) be a
sequence of numbers that is defined as
follows 1. x0 a0, x1 a1, , xk 1 ak
1, where a0, a1, , ak 1 are scalars. 2. For
all n ? k, xn c1xn 1 c2xn 2 ckxn
k, where c1, c2, , ck are scalars. If ck ?
0, the equation in (2) is called a linear
recurrence relation of order k. The equations in
(1) are referred to as the initial conditions of
the recurrence.
71
Ex Solve the Fibonacci recurrence f0 0, f1
1, and fn fn 1 fn 2 for n ? 2
72
(No Transcript)
73
Ex Solve the recurrence relation x0 1, x1 6,
and xn 6xn 1 9xn 2 for n ? 2.
Sol The characteristic equation is ?2 6? 9
0, which has ? 3. xn c13n c2n3n
(c1 c2n)3n. Since 1 x0 c1 and 6
x1 (c1 c2)3, we find that c2 1, so xn
(1 n)3n
74
Systems of Linear Differential Equations
x x(t) is a differentiable function satisfying
a differential equation of the . where
k is a constant, then the general solution is x
Cekt, where C is a constant. If an initial
condition x(0) x0 is specified, we find that C
x0. x x0ekt
75
(No Transcript)
76
(No Transcript)
77
(No Transcript)
78
Theorem 4.41 Let A be an n n diagonalizable
matrix with eigenvalues ?1, ?2, , ?n. Then
the general solution to the system
, where c is an arbitrary
constant vector. If an initial condition x(0) is
specified, then c x(0)

• Discrete Linear Dynamical Systems

Markov chains and the Leslie model of population
growth are examples of discrete linear dynamical
systems. Each can be described by a matrix
equation of the form xk 1 Axk The set
x0, x1, x2, is called a trajectory of the
system.
79
(No Transcript)