Chapter 2: Systems of Linear Equations

1
Chapter 2Systems of Linear Equations
2
2.1 Linear Systems

• Many relationships are linear, meaning that
  effects are proportional to their causes. For
  example Newtons Second Law , Fm a.
• In higher dimensions, such linear relationship is
  expressed by a linear transformation that
  relates a vector to a vector of effects
  , so that

3

• It can be show that a linear transformation
  can be conveniently represented by a matrix.
• In matrix-vector notation the relationship can be
  represented by
• Where is a known m by n matrix, is
  m-vector, and an n-vector.

4
Example Spring-Mass Systems

• Newtons second Las and Hookes Law apply. The
  system is then described by a system of ODEs
• But this can be written in matrix form as

5
2.2 Existence and Uniqueness

• Review of Linear Algebra How many ways are there
  to say a matrix A (n x n) is non-singular?
• A has an inverse i.e., A-1 exists.
• det(A) ? 0
• rank(A) n (rank is the number of linearly
  independent rows it contains)
• A x 0 if and only if x 0
• If a matrix is non-singular then the solution to
  the linear equation A x b is unique. On the
  other hand if the matrix is singular then we
  either have no solution or an infinite number of
  solutions.

6
2.3 Sensitivity and Conditioning

• We now have criteria to describe the existence
  and uniqueness of a solution to a linear system A
  x b , we now consider the sensitivity of the
  solution whenever x is subject to a perturbation.
• But how do we measure such effects?
• We need some notion of size for vectors and
  matrices. We thus extend the concept of absolute
  value to matrices and vectors.

7
Vector Norms

• There are many vector norms out there but for our
  purpose we will only focus on three important
  norms.
• 1-norm
• 2-norm
• 8-norm
• So which one should I use ? Well, it depends on
  the application. The 1-norm and 8-norm are good
  whenever analyzing sensitivity of solutions. The
  2-norm is good for comparing distances of vectors.

8
What about Matrix Norms?

• As in the case for vectors there are many ways to
  define a matrix norm. For our purpose all
  definitions of matrix norms will come from
• Where A is an m x n matrix.
• What does this matrix norm represent?
• It simply represents the maximum stretching it
  does to any vector.

9
Example of Matrix norms

• We have a 1-norm
• Which is just the maximum
• absolute column sum of the matrix.
• We also have an 8-norm
• This is just the maximum absolute row sum of the
  matrix.

10
Properties of Matrix Norms

• A gt 0 if A ? O
• c A c A if A ? O
• A B A B
• A B A B
• A x A x

11
Matrix Condition Number

• By definition the condition number of a
  nonsigular matrix A is given by
• cond(A) A A-1
• by convention if A is singular then cond(A) 8.

12
Properties of the Matrix Condition Number

• For any matrix A, cond(A) 1.
• For the identity matrix, cond(I) 1.
• For any permutation matrix P, cond(P) 1.
• For any matrix A and nonzero scalar c , cond(c A)
  cond(A).
• For any diagonal matrix D diag(di), cond(D)
  (maxdi)/( min di )

13
What does the condition number really tell us?

• The condition number is a good indicator of how
  close is a matrix to be singular. The larger the
  condition number the closer we are to
  singularity.
• It is also very useful in assessing the accuracy
  of solutions to linear systems.
• In practice we dont really calculate the
  condition number, it is merely estimated , to
  perhaps within an order of magnitude. As this
  process is relatively inexpensive.

14
An estimate of the Condition Number

• Choose the norm to either be the 1-norm or the
  8-norm. Then we know if z is the solution to
  A z y, then
• z A-1 y A-1 y
• So that
• z / y A-1 ,
• This bound is achieved optimally chosen by
  some vector y. Thus if we can choose y optimally
  we will have a reasonable estimate for A-1 .

15
Example of Condition Number Estimation
16
Next Time

• More On the Condition Number and more we will
  begin with Linear Systems

17
(No Transcript)