Dynamic programming

1
Dynamic programming

• Binhai Zhu
• Computer Science Department, Montana State
  University

2
Idea

• You have a large problem to solve, you can divide
  the problem into smaller sub-problems
• (1) Solution of a sub-problem might be
  interrelated and might be re-used again (this is
  different from Divide Conquer.
• (2) So it is better to store those smaller
  solutions somewhere.

3
Idea

• You have a large problem to solve, you can divide
  the problem into smaller sub-problems
• (1) Solution of a sub-problem might be
  interrelated and might be re-used again (this is
  different from Divide Conquer.
• (2) So it is better to store those smaller
  solutions somewhere.
• (3) Of course, sometimes it might not work.

4
Example 1. World Series Odds
Two teams A and B play to see who is the first to
win n games. In world series games
n4. Assumption A and B are equally competent,
each has a 50 chance to win any particular game.
5
Example 1. World Series Odds
P(i,j) --- the probability that A needs i extra
games to win and B needs j extra games to win.
6
Example 1. World Series Odds
P(i,j) --- the probability that A needs i extra
games to win and B needs j extra games to win.
(Assume that we talk about team A is to
win/lose.) P(i,j) 1, if i 0 and j
gt 0 P(i,j) 0, if i gt0 and j 0
7
Example 1. World Series Odds
P(i,j) --- the probability that A needs i extra
games to win and B needs j extra games to win.
(Assume that we talk about team A is to
win/lose.) P(i,j) 1, if i 0 and j
gt 0 P(i,j) 0, if i gt0 and j 0 P(i,j) (
P(i-1,j) P(i,j-1) ) /2, if i gt 0 and j gt
0 What is the cost to calculate P(i,j)
recursively?
8
Example 1. World Series Odds

• P(i,j) 1, if i 0 and j gt 0
• P(i,j) 0, if i gt0 and j 0
• P(i,j) ( P(i-1,j) P(i,j-1) ) /2, if i gt 0
  and j gt 0
• What is the cost to calculate P(i,j) recursively?
• Let ij n and T(n) be the cost for calculating
  P(i,j).
• T(1) O(1) c
• T(n) 2T(n-1) O(1) 2T(n-1) d, c,d are
  constants.

9
Example 1. World Series Odds

• Let ij n and T(n) be the cost for calculating
  Pi,j.
• T(1) O(1) c
• T(n) 2T(n-1) O(1) 2T(n-1) d, c,d are
  constants.
• T(n) c2n-1 d(2n-1-1) e O(2n) O(2ij).
• So when n is large, this is not an efficient
  solution.

10
Example 1. World Series Odds

• How to solve this problem with dynamic
  programming?
• (1) Define a Table P-,-
• Odds(i,j)
• for t 1 to ij do
• P0,t 1
• Pt,0 0
• for k 1 to t-1 do
• Pk,t-k(Pk-1,t-kPk,t-k-1)/2
• endfor
• endfor
• return Pi,j // running time?

11
Example 1. World Series Odds
Odds(i,j) for t 1 to ij do P0,t
1 Pt,0 0 for k 1 to t-1
do Pk,t-k(Pk-1,t-kPk,t-k-1)/2
endfor endfor return Pi,j
// running time?
4
1
3
1
? j
2
1
1
1
0
0
0
0
0
0
1
4
3
2
i ?
12
Example 1. World Series Odds
Odds(i,j) for t 1 to ij do P0,t
1 Pt,0 0 for k 1 to t-1
do Pk,t-k(Pk-1,t-kPk,t-k-1)/2
endfor endfor return Pi,j
// running time?
4
1
15/16
3
1
7/8
? j
2
1
3/4
1
1
1/2
0
0
0
0
0
0
1
4
3
2
i ?
13
Example 1. World Series Odds
Odds(i,j) for t 1 to ij do P0,t
1 Pt,0 0 for k 1 to t-1
do Pk,t-k(Pk-1,t-kPk,t-k-1)/2
endfor endfor return Pi,j
// running time?
4
1
1/2
15/16
21/32
13/16
3
1
7/8
1/2
11/16
11/32
? j
2
1
3/4
1/2
5/16
3/16
1
1
1/2
1/4
1/8
1/16
0
0
0
0
0
0
1
4
3
2
i ?
14
Example 2. Matrix Chain Multiplication
Given n matrices M1,M2,,Mn, compute the product
M1M2M3Mn, where Mi has dimension di-1 x di
(i.e., with di-1 rows and di columns), for i
1,,n. Objective?
15
Example 2. Matrix Chain Multiplication
Given n matrices M1,M2,,Mn, compute the product
M1M2M3Mn, where Mi has dimension di-1 x di
(i.e., with di-1 rows and di columns), for i
1,,n. Objective? Fact 1. Given matrices A with
dimension p x q and B with dimension q x r,
multiplication AB takes pqr scalar
multiplications (see handouts).
16
Example 2. Matrix Chain Multiplication
Given n matrices M1,M2,,Mn, compute the product
M1M2M3Mn, where Mi has dimension di-1 x di
(i.e., with di-1 rows and di columns), for i
1,,n. Objective? Fact 1. Given matrices A with
dimension p x q and B with dimension q x r,
multiplication AB takes pqr scalar
multiplications (see handouts). So the objective
is to compute M1M2M3Mn with the minimum number
of scalar multiplications.
17
Example 2. Matrix Chain Multiplication
Problem Parenthesize the product M1M2Mn in a
way to minimize the number of scalar
multiplications. Example. M1 --- 20 x 10
M2 --- 10 x 50 M3 --- 50 x 5
M4 --- 5 x 30 (M1(M2(M3M4))) --- 28500
multiplications (M1((M2M3)M4) --- 10000
multiplications ((M1M2)(M3M4)) --- 47500
multiplications ((M1(M2M3))M4) --- 6500
multiplications (((M1M2)M3)M4) --- 18000
multiplications
18
Example 2. Matrix Chain Multiplication
Problem Parenthesize the product M1M2Mn in a
way to minimize the number of scalar
multiplications. However, exhaustive search is
not efficient. Let P(n) be the number of
alternative parenthesizations of n matrices.
19
Example 2. Matrix Chain Multiplication
Problem Parenthesize the product M1M2Mn in a
way to minimize the number of scalar
multiplications. However, exhaustive search is
not efficient. Let P(n) be the number of
alternative parenthesizations of n matrices. P(n)
1, if n1 P(n) ?k1 to n-1 P(k)P(n-k), if n
2
20
Example 2. Matrix Chain Multiplication
Problem Parenthesize the product M1M2Mn in a
way to minimize the number of scalar
multiplications. However, exhaustive search is
not efficient. Let P(n) be the number of
alternative parenthesizations of n matrices. P(n)
1, if n1 P(n) ?k1 to n-1 P(k)P(n-k), if n
2 P(n) 4n-1/(2n2-n). Ex. n 20, this is gt
228.
21
Example 2. Matrix Chain Multiplication
So lets use dynamic programming. Let mij be the
number of multiplications performed using an
optimal parenthesization of MiMi1Mj-1Mj.
22
Example 2. Matrix Chain Multiplication

• So lets use dynamic programming.
• Let mij be the number of multiplications
  performed using an optimal parenthesization of
  MiMi1Mj-1Mj.
• mii 0
• mij minkmik mk1,j di-1dkdj, 1 i k lt j
  n

23
Example 2. Matrix chain multiplication
Now you see another difference between dynamic
programming and DivideConquer --- dynamic
programming is always bottom-up!
j
1
4
3
2
1
i
0
10000
3500
0
4000
2500
2
7500
0
3
Pass 2
4
0
Pass 1

• mii 0
• mij minkmik mk1,j di-1dkdj, 1 i k lt j
  n

Pass 0
24
Example 2. Matrix chain multiplication
j
m1,4 contains the value of the optimal solution.
1
4
3
2
1
i
0
10000
3500
6500
0
4000
2500
2
7500
0
3
4
0

• mii 0
• mij minkmik mk1,j di-1dkdj, 1 i k lt j
  n

25
Example 3. Optimal Polygon Triangulation
Given a convex polygon Plt v0,v1,,vn-1 gt, a
chord vivj divides P into two polygons
ltvi,vi1,,vjgt and ltvj,vj1,,vigt (assume vnv0
or more generally, vkvk mod n).
v5
v4
v6
v3
v2
v7
v1
v8
v0
26
Example 3. Optimal Polygon Triangulation
Given a convex polygon Plt v0,v1,,vn-1 gt, a
chord vivj divides P into two polygons
ltvi,vi1,,vjgt and ltvj,vj1,,vigt (assume vnv0
or more generally, vkvk mod n).
v5
v4
v6
v3
v2
v7
v1
v8
v0
27
Example 3. Optimal Polygon Triangulation
Given a convex polygon Plt v0,v1,,vn-1 gt, it can
always be divided into n-2 non-overlapping
triangles using n-3 chords.
v5
v4
v6
v3
v2
v7
v1
v8
v0
28
Example 3. Optimal Polygon Triangulation
Given a convex polygon Plt v0,v1,,vn-1 gt, it can
always be divided into n-2 non-overlapping
triangles using n-3 chords.
v5
v4
v6
v3
v2
v7
v1
v8
v0
29
Example 3. Optimal Polygon Triangulation
Given a convex polygon Plt v0,v1,,vn-1 gt, there
could be a lot of triangulations (in fact, an
exponential number of them).
v5
v4
v6
v3
v2
v7
v1
v8
v0
30
Example 3. Optimal Polygon Triangulation
Given a convex polygon Plt v0,v1,,vn-1 gt, we
want to compute an optimal triangulation.
v5
v4
v6
v3
Optimal on what?
v2
v7
v1
v8
v0
31
Example 3. Optimal Polygon Triangulation
Given a convex polygon Plt v0,v1,,vn-1 gt, we
want to compute an optimal triangulation whose
weight is minimized. The weight of a
triangulation is the weight of all its triangles
and the weight of a triangle is the sum of its 3
edge lengths.
v5
v4
v6
v3
v2
v7
v1
v8
v0
32
Example 3. Optimal Polygon Triangulation
Given a convex polygon Plt v0,v1,,vn-1 gt, we
want to compute an optimal triangulation whose
weight is minimized. The weight of a
triangulation is the weight of all its triangles
and the weight of a triangle is the sum of its 3
edge lengths.
v5
v4
The weight of ?v2v4v6 is v2v4v4v6v2v6
v6
v3
v2
v7
v1
v8
v0
33
Example 3. Optimal Polygon Triangulation
Dynamic Programming Solution Let ti,j be the
weight of an optimal triangulation of polygon
ltvi-1,vi,,vjgt.
vj
vk1
vk
vi-1
vi
34
Example 3. Optimal Polygon Triangulation
Dynamic Programming Solution Let ti,j be the
weight of an optimal triangulation of polygon
ltvi-1,vi,,vjgt.
vj
ti,j mink ti,k tk1,j w(?vi-1vkvj)
, iltj ti,i 0
vk1
vk
vi-1
vi
35
Example 3. Optimal Polygon Triangulation
Dynamic Programming Solution Let ti,j be the
weight of an optimal triangulation of polygon
ltvi-1,vi,,vjgt.
vj
ti,j mink ti,k tk1,j w(?vi-1vkvj)
, iltj ti,i 0 Almost identical to the
matrix chain multiplication problem!
vk1
vk
vi-1
vi
36
Example 4. Longest Common Subsequence
Example. XltA,B,C,B,Dgt, YltB,A,D,Cgt, then ltB,Cgt is
a common subsequence of X and Y. The problem has
applications in bioinformatics and
communications.
37
Example 4. Longest Common Subsequence

• Given a sequence Xltx1,x2,,xmgt, another sequence
  Zltz1,z2,,zkgt is a subsequence of X if there
  exists a strictly increasing sequence
  lti1,i2,,ikgt such that xijzj for j 1,2,,k.
• Example. XltA,B,C,B,Dgt, then ZltA,Dgt is a
  subsequence of X.
• Choose i11,i25, then xi1z1A, xi2z2D.

38
Example 4. Longest Common Subsequence

• Given a sequence Xltx1,x2,,xmgt, another sequence
  Zltz1,z2,,zkgt is a subsequence of X if there
  exists a strictly increasing sequence
  lti1,i2,,ikgt such that xijzj for j 1,2,,k.
• Example. XltA,B,C,B,Dgt, then ZltA,Dgt is a
  subsequence of X.
• Choose i11,i25, then xi1z1A, xi2z2D.
• So by name, the LCS of X and Y is a common
  subsequence of X and Y whose length is the
  largest.

39
Example 4. Longest Common Subsequence
Example. XltA,B,C,B,Dgt, YltB,A,C,A,Dgt then
ZltA,C,Dgt is a LCS of X and Y. Can you find
another LCS in this case?
40
Example 4. Longest Common Subsequence
Example. XltA,B,C,B,Dgt, YltB,A,C,A,Dgt then
ZltA,C,Dgt is a LCS of X and Y. Can you find
another LCS in this case? ZltB,C,Dgt is another
LCS, so longest common subsequence is not unique!
41
Example 4. Longest Common Subsequence
Dynamic programming solution. (For simplicity,
lets say we are only interested in finding the
length.) -Define table c-,- -ci,j stores the
length of an LCS of the sequences X1..i and
Y1..j. ci,j 0, if i0 or j0 ci,j
ci-1,j-1 1, if i,jgt0 and xi yj ci,j
max ci-1,j, ci,j-1 , if i,jgt0 and xi ? yj
42
Example 4. Longest Common Subsequence
j
1
4
3
2
0
m ? lengthX n ? lengthY For i 1 to m do
ci,0 ? 0 For j 0 to n do c0,j ? 0 For i
1 to m for j 1 to n if xi
yj then ci,j?ci-1,j-11 else if
ci-1,j ci,j-1 then ci,j ? ci-1,j
else ci,j ? ci,j-1
i
0
0
0
0
0
0
0
1
2
0
0
3
4
0
XltA,C,B,Dgt YltB,A,C,Dgt
43
Example 4. Longest Common Subsequence
j
1
4
3
2
0
m ? lengthX n ? lengthY For i 1 to m do
ci,0 ? 0 For j 0 to n do c0,j ? 0 For i
1 to m for j 1 to n if xi
yj then ci,j?ci-1,j-11 else if
ci-1,j ci,j-1 then ci,j ? ci-1,j
else ci,j ? ci,j-1
i
0
0
0
0
0
0
0
0
1
1
1
1
2
0
0
3
4
0
XltA,C,B,Dgt YltB,A,C,Dgt
44
Example 4. Longest Common Subsequence
j
1
4
3
2
0
m ? lengthX n ? lengthY For i 1 to m do
ci,0 ? 0 For j 0 to n do c0,j ? 0 For i
1 to m for j 1 to n if xi
yj then ci,j?ci-1,j-11 else if
ci-1,j ci,j-1 then ci,j ? ci-1,j
else ci,j ? ci,j-1
i
0
0
0
0
0
0
0
0
1
1
1
1
2
0
0
1
2
2
0
1
1
2
2
3
4
0
1
1
3
2
XltA,C,B,Dgt YltB,A,C,Dgt