Chapter 3 System of Linear Equations 1

1
Chapter 3 System of Linear Equations (1)

• Elementary Matrices
• Rank of a matrix
• rank(A) rank(LA)
• rank(A) dim(Column space of A) dim(Row space
  of A)
• rank(A) max number of l.i. cols of A max
  number of l.i. rows of A.
• Properties
• rank(AQ) rank(A) if Q is invertible
• rank(PA) rank(A) if P is invertible
• rank(PAQ) rank(A) if P, Q are both invertible
• If A is m?n, then rank(A) ? m, rank(A) ? n
• rank(A) rank(At)

2
Chapter 3 System of Linear Equations (2)

• rank(AB) ? rank(A), rank(AB) ? rank(B)
• A is n?n. Then
• A is invertible iff. rank(A) n.
• A is invertible iff. det(A) ? 0.
• A is invertible iff. all eigenvalues of A are
  nonzero.
• Systems of linear equations (Ax b)
• A is m?n (coefficient matrix)
• x is n?1
• b is m?1
• m is the number of equations, n is the number of
  unknowns.
• Solution set x?Fn Ax b

3
Chapter 3 System of Linear Equations (3)

• Solutions of Axb
• Any solution s sp csh, where c is any scalar,
  sp is a particular solution for Axb, and sh is a
  homogeneous solution (Ash0)
• All homogeneous solutions N(LA).
• If rank(A) n, then there is 0 or 1 solution
  (?N(LA) 0).
• If rank(A) lt n, then there is 0 or ? solutions
  (?dim(N(LA)) gt 0)
• Axb has solution iff rank(A) rank(Ab)

4
Chapter 3 System of Linear Equations (4)

• Number of solutions of Axb
• If rank(Ab) rank(A) then
• If rank(A) n there is 1 solution
• If rank(A) lt n there are ? solutions
• If rank(Ab) ? rank(A) then there is no solution
• Special case 1 if A is n?n and invertible, then
  Axb has exactly one solutions.
• (?n rank(A) ? rank(Ab) ? n)
• Special case 2 If A is m?n, mltn, and rank(A)
  m, then Axb has ? solutions.
• (?m rank(A) ? rank(Ab) ? m)

5
Chapter 3 System of Linear Equations (5)

• Solving Axb
• Use Gaussian elimination to transform (Ab) into
  reduced row echelon form.
• Find solution by substitution.
• Finding inverses
• Use Gaussian elimination to transform (AI) into
  (IB), then A-1 B

6
Chapter 4 Determinants

• Some properties
• det(A-1) 1/det(A)
• A is invertible iff det(A) ? 0
• det(AB) det(A)det(B)

7
Chapter 5 Diagonalization (1)

• Motivation
• TV ? V is linear. Can we find ordered basis ?
  for V such that T? is a diagonal matrix?
• A is n?n. Can we find ordered basis ? for Fn
  such that LA? is a diagonal matrix?
• Notes
• In general, LA? Q-1AQ, where the columns of Q
  are vectors in ?.
• If ? is the standard basis, then LA? A
• We want to find Q so that Q-1AQ diagonal
  matrix.

8
Chapter 5 Diagonalization (2)

• Eigenvalues, Eigenvectors, Eigenspaces
• TV ? V is linear. ? is an eigenvalue of T if for
  some x ? 0, T(x) ?x. x is an eigenvector of T
  corresponding to ?.
• A is n?n. ? is an eigenvalue of A if for some x ?
  0, Ax ?x. x is an eigenvector of A
  corresponding to ?.
• For a linear transform TV ? V
• E? Eigenspace corresponding to ? x T(x)
  ?x N(T- ?IV) All eigenvectors of T
  corresponding to ??0.
• For n?n matrix A
• E? Eigenspace corresponding to ? x Ax
  ?x N(A- ?I) All eigenvectors of A
  corresponding to ??0.

9
Chapter 5 Diagonalization (3)

• Properties of eigenvalues
• ? is an eigenvalue of T iff det(T- ?IV)0. ? is
  an eigenvalue of A iff det(A- ?I)0.
• f(t) det(T-tIV) is the characteristic
  polynomial of T.
• Eigenvalues are the roots of the characteristic
  polynomial.
• If dim(V)n, then T has at most n distinct
  eigenvalues If A is n?n, then A has at most n
  distinct eigenvalues
• det(T) product of all eigenvalues of T det(A)
  product of all eigenvalues of A.
• tr(A) sum of all eigenvalues of A
• ? is an eigenvalue of T iff it is an eigenvalue
  of T? for any ordered basis ? for V
• How to find eigenvalues for T
• 1. Find T? for any ordered basis ? for V
• 2. Find eigenvalues by solving det(T? - ?I)
  0
• Similar matrices have same eigenvalues

10
Chapter 5 Diagonalization (4)

• Properties of eigenvectors
• Eigenvectors of T satisfy T(x) ?x for some ??F.
• Eigenvectors of A satisfy Ax ?x for some ??F.
• Eigenvectors are by definition nonzero
• How to find eigenvectors
• 1. Find all eigenvalues of T.
• 2. For each eigenvalue ?, find nonzero
  solutions of T(x) - ?x 0.
• Any nonzero linear combination of eigenvectors
  corresponding to the same eigenvalue ? is also an
  eigenvector corresponding to ?
• Eigenvectors corresponding to distinct
  eigenvalues are l.i.
• Similar matrices do not necessarily have the same
  eigenvectors.
• 1 ? dim(E?) ? multiplicity of ?

11
Chapter 5 Diagonalization (5)

• Properties of eigenspaces
• E? x T(x) ?x N(T- ?IV) All
  eigenvalues of T corresponding to ??0.
• dim(E?) dim(T- ?IV) n rank(T- ?IV)
• 1 ? dim(E?) ? multiplicity of ?
• How to find eigenspaces
• 1. Find all eigenvalues
• 2. For each eigenvalue ?, find N(T- ?IV).

12
Chapter 5 Diagonalization (6)

• Test of diagonalizability
• For linear transforms
• Test 1 TV?V is diagonalizable iff ? an
  ordered basis for V consisting of eigenvectors of
  T.
• Test 2 TV?V is diagonalizable iff
• The characteristic polynomial of T splits and
• For each eigenvalue ? of T, dim(E?)
  multiplicity of ?
• For matrices
• Test 1 A is diagonalizable iff ? an ordered
  basis for Fn consisting of eigenvectors of A.
• Test 2 A is diagonalizable iff
• The characteristic polynomial of A splits and
• For each eigenvalue ? of A, dim(E?)
  multiplicity of ?

13
Chapter 5 Diagonalization (7)

• Invariant spaces and Cayley-Hamilton Theorem
• If f(t) is the characteristic polynomial of T,
  then f(T) T0.
• If f(t) is the characteristic polynomial of A,
  then f(A) 0.

14
Chapter 6 Inner Product Spaces (1)

• Inner product
• V is a vector space over F. x, y are vectors in
  V. ltx,ygt is a scalar in F such that for all x,
  y, z in V and c in F we have
• ltxz,ygt ltx,ygt ltz,ygt
• ltcx, ygt cltx,ygt
• ltx,ygt lty,xgt
• ltx,xgt gt 0 if x ? 0
• Example standard inner product (dot product)
• x, y ? Cn, ltx,ygt yx

15
Chapter 6 Inner Product Spaces (2)

• Inner product space
• A vector space on which an inner product is
  defined.
• Conjugate transpose (Hermitian transpose) of a
  matrix
• A (At)
• Norm of a vector

16
Chapter 6 Inner Product Spaces (3)

• Orthonormal basis
• x, y are orthogonal if ltx,ygt 0
• An ordered basis ? for V is orthonormal if
• all distinct vectors in ? are orthogonal and
• every vector in ? have norm 1.
• Every inner product space has an orthonormal
  basis (can be found from any ordered basis using
  Gram-Schmidt procedure)
• If ? v1, , vn is an orthonormal basis for V,
  then

17
Chapter 6 Inner Product Spaces (4)

• Orthogonal complement
• S ? V. The orthogonal complement of S is
  S? x?V ltx,ygt 0, ? y ? S
• If W is a subspace of V, then dim(W)dim(W?)
  dim(V)
• Orthogonal projection
• Let W be a subspace of V, and y?V. ?! u?W
  and z ?W? such that y u z.

18
Chapter 6 Inner Product Spaces (5)

• Adjoint
• TV?V is a linear transform. Adjoint of T is the
  unique function that satisfies
• ltT(x),ygt ltx,T(y)gt, all x,y in V.
• Properties
• T is linear
• ltx,T(y)gt ltT(x),ygt
• (T-1) (T)-1
• det(T) det(T)
• If ? is an eigenvalue of T then ? is an
  eigenvalue of T
• R(T) (N(T))?
• N(T) (R(T))?
• T? (T?) if ? is an orthonormal basis for
  V
• (LA) LA

19
Chapter 6 Inner Product Spaces (6)

• Least-squares approximation
• Ax b has at most one solution. Assume rank(A)
  n
• Find x0 such that Ax0 ? b
• Solution
• Ax0 is the projection of b on the col space of A
• By geometry, b-Ax0 must be orthogonal to every
  column of A
• Therefore, x0 (AA)-1Ab.

20
Chapter 6 Inner Product Spaces (7)

• Minimal solution
• Ax b has at least one solution.
• Find the solution with the smallest norm
• Solution
• If x0 is any solution of Axb, then the
  projection of x0 on R(LA) is also a solution of
  Ax b
• (?R(LA) (N(LA))?)
• Every solution of Axb has the same projection on
  R(LA). There is exactly one solution in R(LA).
• Therefore the minimal solution is the unique
  solution of Ax b in R(LA). The minimal
  solution s satisfies