Chained Matrix Multiplication

1
Chained Matrix Multiplication

• Lecture 22 Chained Matrix Multiplication
• Chained Matrix Multiplication

• Lecture 21 DP Examples, Knapsack and Floyd
• 0-1 Knapsack
• Floyds shortest paths

Midterm Reminder Next Monday-Wednesday Oct.
27-29th at the Testing Center ends at 500pm on
29th
2
Partial Topic List for Midterm

• Asymptotic notation
• Definitions
• Prove the order of growth of a given function
• Implementation vs. Algorithmic complexity
• Recurrence relations
• Difference equations and dynamic systems
• Linear, Constant Coefficient, Homogeneous
• Change Variables, Geometric forcing functions
• Characteristic Polynomial, Homogeneous solution
• Extended Characteristic Polynomial, Particular
  solution
• Know results and proof for Existence and
  Uniqueness Theorem
• Know results and proofs for Theorems 1-3
• Be able to solve any linear, constant coefficient
  difference equation with geometric forcing
  function
• Know how to use initial condition information to
  completely specify a solution
• Elementary Probability Theory
• Definitions sample space, probability measure,
  random variable, probability mass function,
  average or expected value, etc.
• Know how to use it e.g. average case analysis,
  etc.

3
Partial Topic List for Midterm

• Greedy Algorithms
• Understand general characteristics of a greedy
  algorithm
• Be able to argue whether a given algorithm is
  greedy or not
• Know the making change algorithm
• Know definitions of undirected and directed
  graphs, graphs with weights, and spanning trees,
  minimum spanning trees
• Know Kruskals algorithm
• Know Prims algorithm
• Know definition of a shortest path
• Know Dijkstras algorithm
• Know the knapsack problem and algorithm
• Divide and Conquer Algorithms
• Multiplication
• Binary Search
• Sortingall algorithms we discussed in class
• Median
• Matrix multiplication
• Exponentiation and its role in Public Key
  Cryptography
• Dynamic Programming
• Binomial coefficient

4
Chained Matrix Multiplication

• Multiply a series of matrices
• ABCDEFG
• How you pick the parentheses matters
• ((AB)(CD))((EF)G)
• For n matrices, how many ways are there to
  arrange the parentheses?
• lots, ?(4n/n)

5
Eugene Catalan

• 1814-1894, Belgian
• Number theory
• A series that describes the number
• of ways to dissect a polygon into
• triangles with non-intersecting lines.

6
DP for Matrix Multiplication

• mij optimal solution for computing product of
  matrices i through j.
• m1n optimal solution for all 1n matrices
• Fill in the solution matrix by diagonals, rather
  than rows.
• Combine minimal sub-solutions to obtain minimal
  solution.

7
Bottom Up Algorithm
j1 2 3 4
Best way to multiply 1..4
i1
2 3 4
Best way to multiply 2..3
Best way to multiply 1 through 2
Best way to multiply matrices 1 through1
8
Bottom Up Algorithm
j1 2 3 4
i1
2 3 4
Begin by building all ways to multiply two
adjacent matrices. Then combine those results to
build all ways to multiply 3 matrices. Continue
to 4 matrices.
9
Top Down Algorithm
j1 2 3 4
i1
2 3 4
Try to build the best way to multiply all 4
matrices. To do that, youll need the best ways
to multiply 3 matrices. To get 3 matrices, youll
need the best ways to multiply 2 matrices.
10
Chained Matrix Example
Whats the best way to multiply these four
matrices? (4,4) x (4,20) x (20,10) x (10,3)
First, fill in the diagonal.
320 mult. matrix 1 by matrix 2. 800 mult.
matrix 2 by matrix 3. etc.
11
Chained Matrix Example
Whats the best way to multiply matrices
1..3? (4,4) x (4,20) x (20,10) x (10,3)
Choose (1x2)x3, which takes 320 4x20x10
mults. or 1x(2x3), which takes 800 4x4x10
mults.
12
Chained Matrix Example
Whats the best way to multiply matrices
2..4? (4,4) x (4,20) x (20,10) x (10,3)
Choose (2x3)x4, which requires 800 4x10x3
mults or 2x(3x4), which requires 6004x20x3 mults
13
Chained Matrix Example
Whats the best way to multiply all four
matrices? (4,4) x (4,20) x (20,10) x (10,3)
Its the best of 1x(2x3x4) or (1x2)x(3x4) or
(1x2x3)x4 ? 2x3x4 means the optimal way to
multiply matrices 2,3,4 Question Where is the
number of multiplications for 2x3x4?
14
Chained Matrix Example
Whats the best way to multiply all four
matrices? (4,4) x (4,20) x (20,10) x (10,3)
1x(2x3x4) M2,4 4x4x3 840 48
888 (1x2)x(3x4) M1,2M3,4 4x20x3
320600240 1,160 (1x2x3)x4 M1,3 4x10x3
960120 1,080
15
Analysis and Discussion

• How many diagonals?
• How many elements in diagonal s?
• How many computations to compute each element of
  diagonal s?

16
The Analysis
17
Bottom-up vs. Top-down

• Might compute irrelevant subsolutions
• Manage recursion

18
Top-down Recursive Approach
function fm (i,j) if i j then return 0 m ?
infinity for k 1 to j - 1 do m ? min (m ,
fm(i,k)fm(k1,j) di-1dkdj) return m
19
Top-down Recursive Approach
function fm (i,j) if i j then return 0 m ?
infinity for k 1 to j - 1 do m ? min (m ,
fm(i,k)fm(k1,j) di-1dkdj) return m
Whats the complexity of this algorithm?
20
Top-down Recursive Approach
21
Call Tree
fm(1,4)
fm(1,1) fm(2,4)
fm(1,2) fm(3,4)
fm(1,3) fm(4,4)
fm (2,2) fm(3,4)
fm(1,1) fm(2,2)
fm(3,3) fm(4,4)
fm(1,1) fm(2,3)
fm(1,2) fm(3,3)
fm (2,3) fm(4,4)
fm(3,3) fm(4,4)
fm(2,2) fm(3,3)
fm(2,2) fm(3,3)
fm(1,1) fm(2,2)
22
Call Tree
fm(1,4)
fm(1,1) fm(2,4)
fm(1,2) fm(3,4)
fm(1,3) fm(4,4)
fm (2,2) fm(3,4)
fm(1,1) fm(2,2)
fm(3,3) fm(4,4)
fm(1,1) fm(2,3)
fm(1,2) fm(3,3)
fm (2,3) fm(4,4)
fm(3,3) fm(4,4)
fm(2,2) fm(3,3)
fm(2,2) fm(3,3)
fm(1,1) fm(2,2)
How do you modify fm to avoid recomputing results?
23
Memory Function
function fm-mem (i,j) if i j then return 0 if
mtab i,j gt -1 then return mtabi,j m ?
infinity for k 1 to j - 1 do m ? min (m ,
fm-mem(i,k)fm-mem(k1,j) di-1dkdj
mtabi,j ? m return m
24
Call Tree
fm(1,4)
fm(1,1) fm(2,4)
fm(1,2) fm(3,4)
fm(1,3) fm(4,4)
fm (2,2) fm(3,4)
fm(1,1) fm(2,2)
fm(3,3) fm(4,4)
fm(1,1) fm(2,3)
fm(1,2) fm(3,3)
fm (2,3) fm(4,4)
fm(3,3) fm(4,4)
fm(2,2) fm(3,3)
fm(2,2) fm(3,3)
fm(1,1) fm(2,2)