Instructor: Shengyu Zhang

1
CSC3160 Design and Analysis of Algorithms
Week 5 Dynamic Programming (1)

• Instructor Shengyu Zhang

2
Midterm

• According to the in-class survey
• Time 930am 1030am, Oct 15.
• Place the usual classroom in Shaw College.
• Content the first 5 weeks.
• Others
• Open book.
• There will be a review of the first half of
  semester by our TAs next week.

3
This and next week

• We only have a short class this week due to the
  public holiday.
• Well talk about Dynamic Programming
• A simple but non-trivial method for designing
  algorithms
• Achieve much better efficiency than naïve ones.
• A couple of example problems will be exhibited
  and analyzed.

4
Problem 1 Chain matrix multiplication

5
Suppose we want to multiply four matrices

• Suppose we want to multiply four matrices
  A?B?C?D.
• Dimensions A5020, B201, C110, D10100
• Assume cost (XmnYnl) mnl.
• Try three orders
• A((BC)D) 20110 2010100 5020100
  120,200
• (A(BC))D 10110 502010 5010100
  60,200
• (AB)(CD) 50201 110100 501100
  7,000
• The order matters!
• Question In what order should we multiply them?

6
Key property

• General question We have matrices A1, , An.
• Dimension of Ai mi-1 mi
• Key property solution to a subproblem in an
  optimal solution to the original problem is also
  optimal.
• Suppose we have an best order for A1An
  consider the last step. It must be
  (A1Ai)(Ai1An) for some i?1,,n-1.
• Note that the order for A1Ai and Ai1An
  must also be optimal for those two subproblems.
• Thus we have a recursive method which solves the
  problem.

7
Algorithm

• But what is a best i?
• We dont know Take the min.
• Then we have
• bestcost(1,n) mini(bestcost(1,i)bestcost(i1,n))
  
• bestcost(i,j) the min cost of computing AiAj
• Doing this recursively has complexity
• T(1,n) ?i(T(1,i) T(i1,n))
• Exponential!

8
Wait a minute

• Lets look at the recursion again.
• P(i,j) Computing the min cost of multiplication
  AiAj

1 2 3 4
n

9
What did we observe?

• Simple recursion calculates one small problem
  many times!
• Since well calculate it many times, why not just
  do it once and store the result for later
  reference?
• Thats dynamic programming.
• We first compute the small problems and store the
  answers
• Then compute the large problems using the stored
  results of smaller subproblems.

10
Algorithm
For the previous example s1 bestcost(A1A2),
bestcost(A2A3), bestcost(A3A4) s2
bestcost(A1A2A3), bestcost(A2A3A4) s3
bestcost(A1A2A3A4).

• for i 1 to n
• C(i, i) 0
• for s 1 to n - 1 // s step length
• for i 1 to n - s
• j i s
• C(i, j) min C(i, k) C(k 1, j) mi-1mkmj
  i k lt j
• return C(1, n)

Best cost of AiAk
Best cost of Ak1Aj
Cost of XY, where X AiAk , Y Ak1Aj
s
i
jis
11
Complexity

• for i 1 to n
• C(i, i) 0
• for s 1 to n - 1
• for i 1 to n - s
• j i s
• C(i, j) minC(i, k)C(k 1, j)mi-1mkmj
  ikltj -O(n)
• return C(1, n)
• Total O(n2)O(n) O(n3)
• Much better than the exponenetial!

12
Next week

• More examples.