Dynamic programming

1
Dynamic programming

• Binhai Zhu
• Computer Science Department, Montana State
  University

2
Idea

• You have a large problem to solve, you can divide
  the problem into smaller sub-problems
• (1) Solution of a sub-problem might be
  interrelated and might be re-used again (this is
  different from Divide Conquer.
• (2) So it is better to store those smaller
  solutions somewhere.

3
Idea

• You have a large problem to solve, you can divide
  the problem into smaller sub-problems
• (1) Solution of a sub-problem might be
  interrelated and might be re-used again (this is
  different from Divide Conquer.
• (2) So it is better to store those smaller
  solutions somewhere.
• (3) Of course, sometimes it might not work.

4
Example 1. World Series Odds
Two teams A and B play to see who is the first to
win n games. In world series games
n4. Assumption A and B are equally competent,
each has a 50 chance to win any particular game.
5
Example 1. World Series Odds
P(i,j) --- the probability that A needs i extra
games to win and B needs j extra games to win.
6
Example 1. World Series Odds
P(i,j) --- the probability that A needs i extra
games to win and B needs j extra games to win.
(Assume that we talk about team A is to
win/lose.) P(i,j) 1, if i 0 and j
gt 0 P(i,j) 0, if i gt0 and j 0
7
Example 1. World Series Odds
P(i,j) --- the probability that A needs i extra
games to win and B needs j extra games to win.
(Assume that we talk about team A is to
win/lose.) P(i,j) 1, if i 0 and j
gt 0 P(i,j) 0, if i gt0 and j 0 P(i,j) (
P(i-1,j) P(i,j-1) ) /2, if i gt 0 and j gt
0 What is the cost to calculate P(i,j)
recursively?
8
Example 1. World Series Odds

• P(i,j) 1, if i 0 and j gt 0
• P(i,j) 0, if i gt0 and j 0
• P(i,j) ( P(i-1,j) P(i,j-1) ) /2, if i gt 0
  and j gt 0
• What is the cost to calculate P(i,j) recursively?
• Let ij n and T(n) be the cost for calculating
  P(i,j).
• T(1) O(1) c
• T(n) 2T(n-1) O(1) 2T(n-1) d, c,d are
  constants.

9
Example 1. World Series Odds

• Let ij n and T(n) be the cost for calculating
  Pi,j.
• T(1) O(1) c
• T(n) 2T(n-1) O(1) 2T(n-1) d, c,d are
  constants.
• T(n) c2n-1 d(2n-1-1) e O(2n) O(2ij).
• So when n is large, this is not an efficient
  solution.

10
Example 1. World Series Odds

• How to solve this problem with dynamic
  programming?
• (1) Define a Table P-,-
• Odds(i,j)
• for t 1 to ij do
• P0,t 1
• Pt,0 0
• for k 1 to t-1 do
• Pk,t-k(Pk-1,t-kPk,t-k-1)/2
• endfor
• endfor
• return Pi,j // running time?

11
Example 1. World Series Odds
Odds(i,j) for t 1 to ij do P0,t
1 Pt,0 0 for k 1 to t-1
do Pk,t-k(Pk-1,t-kPk,t-k-1)/2
endfor endfor return Pi,j
// running time?
4
1
3
1
? j
2
1
1
1
0
0
0
0
0
0
1
4
3
2
i ?
12
Example 1. World Series Odds
Odds(i,j) for t 1 to ij do P0,t
1 Pt,0 0 for k 1 to t-1
do Pk,t-k(Pk-1,t-kPk,t-k-1)/2
endfor endfor return Pi,j
// running time?
4
1
15/16
3
1
7/8
? j
2
1
3/4
1
1
1/2
0
0
0
0
0
0
1
4
3
2
i ?
13
Example 1. World Series Odds
Odds(i,j) for t 1 to ij do P0,t
1 Pt,0 0 for k 1 to t-1
do Pk,t-k(Pk-1,t-kPk,t-k-1)/2
endfor endfor return Pi,j
// running time?
4
1
1/2
15/16
21/32
13/16
3
1
7/8
1/2
11/16
11/32
? j
2
1
3/4
1/2
5/16
3/16
1
1
1/2
1/4
1/8
1/16
0
0
0
0
0
0
1
4
3
2
i ?
14
Example 2. Matrix Chain Multiplication
Given n matrices M1,M2,,Mn, compute the product
M1M2M3Mn, where Mi has dimension di-1 x di
(i.e., with di-1 rows and di columns), for i
1,,n. Objective?
15
Example 2. Matrix Chain Multiplication
Given n matrices M1,M2,,Mn, compute the product
M1M2M3Mn, where Mi has dimension di-1 x di
(i.e., with di-1 rows and di columns), for i
1,,n. Objective? Fact 1. Given matrices A with
dimension p x q and B with dimension q x r,
multiplication AB takes pqr scalar
multiplications (see handouts).
16
Example 2. Matrix Chain Multiplication
Given n matrices M1,M2,,Mn, compute the product
M1M2M3Mn, where Mi has dimension di-1 x di
(i.e., with di-1 rows and di columns), for i
1,,n. Objective? Fact 1. Given matrices A with
dimension p x q and B with dimension q x r,
multiplication AB takes pqr scalar
multiplications (see handouts). So the objective
is to compute M1M2M3Mn with the minimum number
of scalar multiplications.
17
Example 2. Matrix Chain Multiplication
Problem Parenthesize the product M1M2Mn in a
way to minimize the number of scalar
multiplications. Example. M1 --- 20 x 10
M2 --- 10 x 50 M3 --- 50 x 5
M4 --- 5 x 30 (M1(M2(M3M4))) --- 28500
multiplications (M1((M2M3)M4) --- 10000
multiplications ((M1M2)(M3M4)) --- 47500
multiplications ((M1(M2M3))M4) --- 6500
multiplications (((M1M2)M3)M4) --- 18000
multiplications
18
Example 2. Matrix Chain Multiplication
Problem Parenthesize the product M1M2Mn in a
way to minimize the number of scalar
multiplications. However, exhaustive search is
not efficient. Let P(n) be the number of
alternative parenthesizations of n matrices.
19
Example 2. Matrix Chain Multiplication
Problem Parenthesize the product M1M2Mn in a
way to minimize the number of scalar
multiplications. However, exhaustive search is
not efficient. Let P(n) be the number of
alternative parenthesizations of n matrices. P(n)
1, if n1 P(n) ?k1 to n-1 P(k)P(n-k), if n
2
20
Example 2. Matrix Chain Multiplication
Problem Parenthesize the product M1M2Mn in a
way to minimize the number of scalar
multiplications. However, exhaustive search is
not efficient. Let P(n) be the number of
alternative parenthesizations of n matrices. P(n)
1, if n1 P(n) ?k1 to n-1 P(k)P(n-k), if n
2 P(n) 4n-1/(2n2-n). Ex. n 20, this is gt
228.
21
Example 2. Matrix Chain Multiplication
So lets use dynamic programming. Let mij be the
number of multiplications performed using an
optimal parenthesization of MiMi1Mj-1Mj.
22
Example 2. Matrix Chain Multiplication

• So lets use dynamic programming.
• Let mij be the number of multiplications
  performed using an optimal parenthesization of
  MiMi1Mj-1Mj.
• mii 0
• mij minkmik mk1,j di-1dkdj, 1 i k lt j
  n

23
Example 2. Matrix chain multiplication
Now you see another difference between dynamic
programming and DivideConquer --- dynamic
programming is always bottom-up!
j
1
4
3
2
1
i
0
10000
3500
0
4000
2500
2
7500
0
3
Pass 2
4
0
Pass 1

• mii 0
• mij minkmik mk1,j di-1dkdj, 1 i k lt j
  n

Pass 0
24
Example 2. Matrix chain multiplication
j
m1,4 contains the value of the optimal solution.
1
4
3
2
1
i
0
10000
3500
6500
0
4000
2500
2
7500
0
3
4
0

• mii 0
• mij minkmik mk1,j di-1dkdj, 1 i k lt j
  n

25
Example 3. Optimal Polygon Triangulation
Given a convex polygon Plt v0,v1,,vn-1 gt, a
chord vivj divides P into two polygons
ltvi,vi1,,vjgt and ltvj,vj1,,vigt (assume vnv0
or more generally, vkvk mod n).
v5
v4
v6
v3
v2
v7
v1
v8
v0
26
Example 3. Optimal Polygon Triangulation
Given a convex polygon Plt v0,v1,,vn-1 gt, a
chord vivj divides P into two polygons
ltvi,vi1,,vjgt and ltvj,vj1,,vigt (assume vnv0
or more generally, vkvk mod n).
v5
v4
v6
v3
v2
v7
v1
v8
v0
27
Example 3. Optimal Polygon Triangulation
Given a convex polygon Plt v0,v1,,vn-1 gt, it can
always be divided into n-2 non-overlapping
triangles using n-3 chords.
v5
v4
v6
v3
v2
v7
v1
v8
v0
28
Example 3. Optimal Polygon Triangulation
Given a convex polygon Plt v0,v1,,vn-1 gt, it can
always be divided into n-2 non-overlapping
triangles using n-3 chords.
v5
v4
v6
v3
v2
v7
v1
v8
v0
29
Example 3. Optimal Polygon Triangulation
Given a convex polygon Plt v0,v1,,vn-1 gt, there
could be a lot of triangulations (in fact, an
exponential number of them).
v5
v4
v6
v3
v2
v7
v1
v8
v0
30
Example 3. Optimal Polygon Triangulation
Given a convex polygon Plt v0,v1,,vn-1 gt, we
want to compute an optimal triangulation.
v5
v4
v6
v3
Optimal on what?
v2
v7
v1
v8
v0
31
Example 3. Optimal Polygon Triangulation
Given a convex polygon Plt v0,v1,,vn-1 gt, we
want to compute an optimal triangulation whose
weight is minimized. The weight of a
triangulation is the weight of all its triangles
and the weight of a triangle is the sum of its 3
edge lengths.
v5
v4
v6
v3
v2
v7
v1
v8
v0
32
Example 3. Optimal Polygon Triangulation
Given a convex polygon Plt v0,v1,,vn-1 gt, we
want to compute an optimal triangulation whose
weight is minimized. The weight of a
triangulation is the weight of all its triangles
and the weight of a triangle is the sum of its 3
edge lengths.
v5
v4
The weight of ?v2v4v6 is v2v4v4v6v2v6
v6
v3
v2
v7
v1
v8
v0
33
Example 3. Optimal Polygon Triangulation
Dynamic Solution Let ti,j be the weight of an
optimal triangulation of polygon ltvi-1,vi,,vjgt.
vj
vk1
vk
vi-1
vi
34
Example 3. Optimal Polygon Triangulation
Dynamic Solution Let ti,j be the weight of an
optimal triangulation of polygon ltvi-1,vi,,vjgt.
vj
ti,j mink ti,k tk1,j w(?vi-1vkvj)
, iltj ti,i 0
vk1
vk
vi-1
vi
35
Example 3. Optimal Polygon Triangulation
Dynamic Solution Let ti,j be the weight of an
optimal triangulation of polygon ltvi-1,vi,,vjgt.
vj
ti,j mink ti,k tk1,j w(?vi-1vkvj)
, iltj ti,i 0 Almost identical to the
matrix chain multiplication problem!
vk1
vk
vi-1
vi