???? (Dynamic Programming: DP)

1
????(Dynamic Programming DP)
??? ?????????????? gongxj_at_tju.edu.cn http//cs.tju
.edu.cn/faculties/gongxj/course/algorithm/
http//groups.google.com/group/algorithm-practice
-team
2
Outline

• What is the DP
• Definition
• Solutions
• Typical applications
• 0/1 Knapsack
• Matrix Multiplication Chains
• All Pairs Shortest Paths
• Maximum Non-crossing Subset nets MNS
• Longest Common Subsequence
• Advanced topics
• Hidden Markov Model
• Mathematics optimization

3
??????????
?????????1???7?????
2
7
5
1
1
2
6
1
3
7
4
5
1
6
6
8
4

• ???(1-gt3-gt5-gt7)????????????7????.??, (3-gt5-gt7)
• ??????????2,3,4??????,???????????

4
???????
?????????s???e?????
2
7
5
1
1
2
6
1
3
7
4
5
1
6
6
8
4

• ?c(i)???i?????e??????, A(i)??i???????,?
• c(s)?????????
• c(e)0
• c(i)minj ? A(i)c(j)cost(i, j) sltilte

5
?????
2
7
5
1
1
2
6
1
3
7
4
5
1
6
6
8
4

• ???c(7)0
• ????c(6),,c(1)
• c(6)1
• c(5)2
• c(4)8c(6)9
• c(3)min1c(5),5c(6)3,63
• c(2)min7c(5),6c(6)9,77
• c(1)min1c(2),4c(3),6c(4)8,7,157
• ????????
• c(i)??1???i???????
• ???c(1)0??

6
What is the Dynamic Programming
A method of solving complex problems by breaking
them down into simpler steps.

• The term was originally used in the 1940s by
  Richard Bellman 
• DP refers specifically to nesting smaller
  decision problems inside larger decisions
• Dynamic was chosen by Bellman because it sounded
  impressive, not because it described how the
  method worked.
• Programming referred to the use of the method to
  find an optimal program
• DP  is both a mathematical optimization method,
  and a computer programming method.
• simplifying a complicated problem by breaking it
  down into simpler subproblems in
  a recursive manner

7
Where is the DP applicable

• Optimal substructure
• A problem is said to have optimal substructure if
  an optimal solution can be constructed
  efficiently from optimal solutions to its
  subproblems.
• Also is called Principle of Optimality by Bellman
  
• Overlap subproblems
• A problem is said to have overlapping
  subproblems if the problem can be broken down
  into subproblems which are reused several times.
• Subproblems must be only 'slightly' smaller than
  the larger problem
• A constant additive factor
• A multiplicative factor  -gt D C

8
How to solve a DP problem

• Define an value function of states
• Write out the recursive equation of value
• Solve the recursive equation
• Top-down approach
• Bottom-up approach
• Traceback the optimal solution

9
0/1????(0/1 Knapsack)

• ????
• ?n?????1,,n ,?????c????,????????????
• ??
• n, c ????????????
• p1,p2, , pn????????
• w1,w2, ,wn???????
• ????
• 0/1?????????1,,n?????(x1, ,xn?0/1??), ???????
• ???????????????????
• ??????????????1,??????2,,n???,????????,?????

10
???????????
?? n5,c10 p6,3,5,4,6 w2,2,6,5,4

• (1,1,0,0,1)?????,????1,2,5
• ??1?????2,?????8
• ?????n4,c8,???2,3,4,5
• (1,0,0,1),????1?5,????????
• ????????????

11
????????

• ?????????????1,???????????????????
• ?f(i, y)??????y,???i,,n,????????,????????
• f(1,c)maxf(2,c), f(2,c-w1)p1
• ?????????(??),??2??????????.
• ????????y, ??i,,n ????????.

12
0/1???????????
?? n3, c116 w100,14,10 p20,18,15,

• ????1(x1 1),??????r16 x2,x3 1,0
  ????????,??18,?????201838
• ????1(x1 0)????????????,???????116,1,1???????,?
  ?33
• ??????38, ???33 ??????1,1,0, ????38

13
0/1????????

• ?f(i,y) ?????y,??i,i1,,n ??????,?????????????

• ???????????
• f(1,c)maxf(2,c),f(2,c-w1)p1
• ??
• ???f(n, )??((15-1)?)
• ????(15-2)?????f(i,) ( in-1,n-2,,2 ),
• ???? f(1,c)

14
0/1?????15-4
?? n3, w100,14,10, p20,18,15, c116

• ???

• ?????(15-2),??

• ?????
• f(1,116 )maxf(2,116),f(2,116-w1)p1
• maxf(2,116),f(2,16)20
• max33,3838

15
0/1?????15-4 (??2)

• ?????traceback????x1,?,xn?
• f(1,c) f(2,c)gtx10??x11,cc-w1?
• x2 f(2,c)f(3,c)gt x20??x21,cc-w2
• Xi f(i,c)f(i1,c)gt xi0??xi1,cc-wi
• ???,
• f(2,116)33?f(1,116),??x11. ????116-10016,
• f(2,16)18,f(3,16) 14?f(2,16),??x21
• ??r16-142,??????3,??x3 0

16
??????

• ????
• ??????(Decision sequences)
• ??????(Problem states)
• ??????(Principle of optimal)
• ????????????(Recurrence equation)
• ??(traceback)?????(Optimal solution)
• ?????
• ???????????????????, ?????????,???????????????????
  ??????????????
• ???????????????????,???????????????????????
• ??????????????,????????????

17
??????

• 0/1????
• ?????
• ????
• ????????

18
0/1????-????

• 1. ????
• 2. ?????????
• 3. ????

19
????

• ??????????t(n)
• t(1)a
• t(n)2t(n-1)b (ngt1),??a,b???
• ????t(n)T(2n)

20
?????15-5
n5, c10 p6,3,5,4,6 w2,2,6,5,4 ,?f (1,10).

• ????f (1,10),????F(1,10)
• ??F(i1,y), F(i1,y-wi)pi
• ??????????????
• ???????y????
• ?j??????F(j,)
• ?????F(1,10),?????????,????F(2,10)?F(2,8)?

10
10
8
10
8
8
6
10
8
4
2
8
6
2
0
10
4
5
8
8
3
2
2
6
3
1
0
21
?????15-5(?)

• ???????,???????????26?????
• ???????,?f(3,8),f(4,8), f(4,2), f(5,8), f(5,3),
  f(5,2)
• ???????????,????????20,?????????????

10
10
8
10
8
8
6
10
8
4
2
8
6
2
0
10
4
5
8
8
3
2
2
6
3
1
0
22
W?????????(1)

• ????????f iy?????f ??????f(i,y) ?????
• ?????T(nc)??
• ???????T(nc)-?????? c?????????, c??????????log2c

23
W?????????(2)

• ??Traceback?fiy?????xi?
• Traceback?????T(n).

????????? 1)????????? 2)?????c
???, ??cgt2n,???????O(n2n).
24
???????

• ????i,?f(i, y)???????(y, f(i, y))?????P(i)?.
• P(i)???(y,f(i, y)) ?y?????.
• ??(a,b)???????i,?,n??????a,??????b.

???????????P(i1)??P(i)?
25
???????

• ?Q(s,t)wisltc, (s-wi,t-pi)?P(i1),?
• Q???xi1???
• P(i1)???xi0???
• ??P(i)P(i1)?Q
• ??P(i1) ?Q
• ???????????(??)
• ?(a,b)?(u,v)???P(i1)?Q???,?au?bltv,??(a,b)?(u,v)
  ??
• (a,b)??????????a?????b,?(u,v)?????????????u???
  ??v,??b
• ??????????
• ???P(i)???wgtc???(w,v)

26
????15-6
?n5,c10 p6,3,5,4,6 w2,2,6,5,4
?f(1,10)

• P(5)(0,0), (4,6) ,Q(5,4),(9,10)
• ???P(4)(0,0),(4,6),(9,10)
• ?? Q(6,5),(10,11)
• ??? P(3)(0,0),(4,6),(9,10),(10,11)
• ?? Q(2,3),(6,9)
• ??? P(2)(0,0),(2,3),(4,6),(6,9),(9,10),(10,11)
• ?? Q(2,6),(4,9),(6,12),(8,15)
• ???P(1)(0,0),(2,6),(4,9),(6,12),(8,15)
• ??????15
• ?????1,1,0,0,1

27
???????

• P(i) ?????????P(i1)??????2?.??P(n)2,??
• P(i)?????????2(n-i1)
• ??Q?O(P(i1))???,??P(i1) ?Q??O(P(i1)Q)O(
  2P(i1))???.????P(i)?O(2n-i1)??
• ????P(i) ????????O(2n)?
• ????????????2n??

28
?????(Matrix multiPlication Chains MPC)

• mn??A?np??B???????? mnp
• ??????A(mn),B(np)?C(pq)????????
• ????????A??B????C,??????????(AB)C
• ??????A (BC)
• ???????????????????,???????????
• ????????mp(nq)
• ????????nq(mp)

29
MPC ??

• ??A?1001??,B?1100??,C?1001??,(AB) C?????
• 1001100100100120000
• ? A (BC) ??????1100110011200
• ??q???????????O(2q)????????
• ???????,?????????

30
MPC?????

• ?M(i,j)???Mi,Mj (ij)???.????????????M(i,
  k)?M(k1,j),??????
• ???M(i,j)??????????M(i,k)?M(k1,j)??????
• ??5???????,?????r (10, 5, 1, 10, 2,
  10),?M1?105???,??
• ????????(M1M2)((M3M4)M5))190
• ??M(3,5)M3M4M5????????
• (M3M4)M5),?????5???????????????

31
MPC?????(?)

• ?c(i,j)???M(i,j)??????(???),??????,???????
• c(i,j)minikltjc(i,k)c(k1,j)rirk1rj1
• c(i,j)???
• c(i,i)0
• c(i,i1)riri1ri2
• ?kay(i,j)??c(i,j)??????k,?
• Kay(i,i1)i
• ????
• MPC??????
• ???????c(1,q)
• ?kay(i, j)???????????

32
MPC ?15-13

• ?q 5?r (10 , 5 , 1 , 10 , 2 , 10)??MPC

M(1,5)M(1,2)M(3,5)
M(3,5)M(3,4)M(5,5)
c(1,5)min c(1,1)c(2,5)500,
c(1,2)c(3,5)100,
c(1,3)c(4,5)1000,
c(1,4)c(5,5)200
c(1,5)190,kay(1,5)2
c(1,4)90,kay(1,4)2
c(1,3)150,kay(1,3)2
c(2,5)minc(2,2)c(3,5)50,
c(2,3)c(4,5)500,
c(2,4)c(5,5)100
c(2,5)90,kay(2,5)2
c(3,5)minc(3,3)c(4,5)100,
c(3,4)c(5,5)20 min300,40
c(3,5)40,kay(3,5)4
c(2,4)minc(2,2)c(3,4)10,
c(2,3)c(4,4)100 min30,150
c(2,4)30,kay(2,4)2
33
MPC ????
//??c(i,j) ????? If cijgt0 return cij

• ???C?,r ???????,kay???????.
• ??C??c(i j) ????kayi jkay (i,j).
• ??Traceback ????C?????kay??????????????
• ??ci,j???????

ciju
34
A deep look at the structures of MPC
35
MPC ????

• ??c ??????????????????.??s 2, 3, , q-1 ?????
  c(i,is),??c ?kay ??????.??????????

36
MPC ??????
?q 5?r (10 , 5 , 1 , 10 , 2 , 10)??MPC
1 2 3 4 5
1 0 50 150 90 190
2 0 50 30 90
3 0 20 40
4 0 200
5 0
37
MPC ????
38
?????

• ????O(q3) .
• ??C(i, is)? T(s)??.
• ?s2,q-1,? ??q-s?C(i,is),??????T((q-s)s).
• ????????T(q3)
• ???kay ???????15-6??Traceback ?????????????

39
????(All Pairs Shortest Paths APSP)

• ??????G????,???????????(cost),???????????(???)???
  ???????????
• ??????(i, j),???i ?j ??????,???????????i ?j ?????
• ?????????
• ???????????

40
APSP ?15-15

• ??15-4??????1???3????
• 1) 1,2,5,3
• 2) 1,4,3
• 3) 1,2,5,8,6,3
• 4) 1,4,6,3
• ?????,?????????1 0,28, 9,2 7.????3)
  ??????1???3??????

41
APSP ?????

• ????1?n??.
• ??c(i,j,k)i?j??????????k??????.
• c(i,j,0)cost(i,j) if lti,jgt?G else c(i,j,0)8
• c(i,i,k)0 for all k
• c(i, j, n)????i?j??????
• ????c(i,j,k)?c(i,j,k-1)???????

42
APSP???

• ????kgt0,
• c(i,j,k)minc(i, j, k-1), c(i, k, k-1)
• c(k, j, k-1)
• ??
• c(i,k,k-1)c(i,k,k)
• c(k,j,k)c(k,j,k-1)
• ?????????????,???c(i,j,n)??????.??????.????c
  ???????O(n3).
• ??????????15-5???

43
APSP????

• ?C(k)????(c(i,j,k))i,j1,,n
• ??C(0)(c(i, j)),???????.
• ??????C(k)
• ?k?k????
• C(k) (i,k)C(k-1)(i,k)
• C(k) (k,j)C(k-1)(k,j).
• ?k?k??????
• C(k)(i,j)maxC(k-1)(i, j), C(k-1)(i,
  k)C(k-1)(k,j)

44
APSP???????(1)
Kayij??i?j?????????????,???????????????
45
APSP???????(2)
46
APSP ?15-17

• ?15-6a ?????????a,15-6b ?????15-9?????c ??,15-6c
  ????kay????15-6c ??kay ?,???1?5???????1?kay15
  4?????????4?5?????,??kay450,???4?5???????????
  ?1?4???????kay143???????,????1?5??????1,2,3,
  4,5?

47
APSP??(1)
48
APSP??(2)

• C(0)(i, j)c(i, j)
• C(1)(3,2)minC(0)(3,2),C(0)(3,1)C(0)(1,2)7
• C(2)(1,3)minC(1)(1,3),
• C(1)(1,2)C(1)(2,3)
• min11,426
• C(3)(1,2)min4,674
• C(3)(2,1)min6,235

49
Maximum Noncrossing Subset of NetsMNS
1 2 3 4 5 6
7 8 9 10
i
Ci
1 2 3 4 5 6
7 8 9 10

• ??i ?????Ci??,???(i,Ci)
• (i,Ci)?(j,Cj)???????,?(iltj) ?(CiltCj)
• ??
• MNS(i,j)(u,Cu) ?u i Cu j?????????????
• size(i,j) MNS(i,j)
• ???MNS(5,7)(4,2)(5,5) size(5,7)2

50
MNS????
51
MNS ??1
1 2 3 4 5 6
7 8 9 10
i
Ci
1 2 3 4 5 6
7 8 9 10

• ????
• size(1,j)0, j1,2,3,4,5,6,7
• size(1,j)1,j8,9,10
• ? C2 7,??size(2,j)0 for j1,,6
• size(2,7)size(1,6)11
• size(2,8)maxsize(1,8),size(1,6)11

52
MNS ??2
53
MNStaceback

• Nets ?i ??
• ??size(i,j)!size(i-1,j) ?MNS(i,j)??net i
• ?jCi-1??????2.

54
LCS Longest Common Subsequences

• Sequence similarity measurement
• Substring search
• The number of changes turning one to another
• Longest common subsequences
• Given two sequences x1 . . m and y1 . . n,
  find a longest subsequence common to them both
• Brute force approach O(n2m)
• DP approach O(nm)

55
(No Transcript)
56
Definition 1 subsequence

• Definition 1 Given a sequence Xx1x2...xm,
  another sequence Zz1z2...zk is a subsequence of
  X if there exists a strictly increasing sequence
  i1i2...ik of indices of X such that for all
  j1,2,...k, we have xijzj.
• Example 1 If Xabcdefg, Zabdg is a subsequence
  of X. Xabcdefg,Zab d g

57
Definition 2 common subsequence

• Definition 2 Given two sequences X and Y, a
  sequence Z is a common subsequence of X and Y if
  Z is a subsequence of both X and Y.
• Example 2 Xabcdefg and Yaaadgfd. Zadf is a
  common subsequence of X and Y.
• Xabc defg
• Yaaaadgfd
• Za d f

58
Definition 3 longest common subsequence

• Definition 3 A longest common subsequence of X
  and Y is a common subsequence of X and Y with the
  longest length. (The length of a sequence is the
  number of letters in the sequence.)
• Longest common subsequence may not be unique.
• Example abcd
• acbd
• Both acd and abd are LCS.

59
Optimal substructure of an LCS

• If
• Let Xx1x2...xm, and Yy1y2...yn be two
  sequences, and
• Zz1z2...zk be any LCS of X and Y.
• Then
• If xmyn, then zkxmyn and Z1..k-1 is an LCS
  of X1..m-1 and Y1..n-1.
• If xm ?yn, then zk?xm implies that Z is an LCS of
  X1..m-1 and Y.
• If xm ?yn, then zk?yn implies that Z is an LCS of
  X and Y1..n-1.

60
The recursive equation

• Let ci,j be the length of an LCS of X1...i
  and Y1...j.
• ci,j can be computed as follows
• Computing the length of an LCS
• There are n?m ci,js. So we can compute them in
  a specific order.

61
Algorithm of LCS
1. for i1 to m do 2. ci,00 3.
for j0 to n do 4. c0,j0 5. for i1
to m do 6. for j1 to n do 7. 8.
if xI yj then 9.
ci,jci-1,j-11 10
bi,j1 11. else if
ci-1,jgtci,j-1 then 12.
ci,jci-1,j 13.
bi,j2 14. else
ci,jci,j-1 15.
bi,j3 14 bi,j stores the
directions. 1diagnal, 2-up, 3-forward.
62
Example 1 of an LCS
XBDCABA and YABCBDAB
63
Constructing an LCS
PrintLCS(b,X,i,j) 1. im 2. jn 3. if i0 or
j0 then exit 4. if bi,j1 then
ii-1 jj-1 print
xi 5. if bi,j2 ii-1 6. if
bi,j3 jj-1 7. Goto Step 3.
The time complexity O(nm).
64
i
Yj
B
B
A
C
D
Xi
0
0
0
0
0
0
0
A
1
0
1
0
0
0
1
B
2
0
1
2
1
1
1
3
C
0
1
1
2
2
2
3
4
B
0
1
1
2
2
LCS (reversed order)
B
C
B
B C B
LCS (straight order)
65
Hidden Markov Model

• HMM ? (OI , OO, A, B, p )
• OI i1,....iNset of state
• OO v1,...,vMset of observation
• A aij,aij p(It1 ij It ii) transition
  probability
• B bik,bik p(Ot vk It ii) symbol
  probability
• p pi, pi p(I1 ii) init state
  distribution
• Where
• Ilt i1,...iT gt is the state sequence
• Oltv1,...,vTgt is the output sequence

66
Three Problems of HMM

• Evaluation
• Given model ? and output sequence O, computing
  p(O ?)
• Decoding
• Given model ? and output sequence O, to find
  the state sequence X such that
• maximize p(O, I ?)
• Learning
• Given output sequence O, to find the model ?
  such that
• maximize p(O, I ?)

67
????

• ??????????
• ????????????(??)
• ??
• 0/1????
• ?????
• ??????????
• MNS
• ??????????????

68
????(1)

• 0/1????
• n4,c20,w(10,15,6,9)
• p(2,5,8,1)
• ??????P(1),P(2),P(3)?????
• ????
• ???????????????????????????
• O(min2n,nS1inpi ,nc)
• ??P(i) 1Sijnpj

69
????(2)

• ???????Ss1,s2,...,sn ?n??????,????????M????S??
  ?
• ????NP-????,????????????

70
??(3)

• r(10,20,50,1,100),????????????????

71
??(4),??19

• T(i,j)??i ??j???????(j1,2)
• C(i,j)??i ??j???????(j1,2)
• ????????,???????1??????2
• ?????t?????????????
• cost(i,j)??1?i??j??????????
• cost(1,j)8 jltT(1,1)
• C(1,1) T(1,1)ltjltT(1,2)
• minC(1,1),C(1,2) T(1,2)ltj
• cost(i,j)mincost(i-1,j-T(i,1))C(i,1),
• cost(i-1,j-T(i,2))C(i,2)
• 8?????j??????i???

72
??(4)

• ??????????????(backward)
• ??????????????
• ?n3, T(2,1,43,2,1)
• C(1,5,22,3,4),t8
• ????????????????????????????????
• ????????

73
??(5),??20

• ??????n?????????????w(i,j)???j?????i???c(i,j)?
  ??j?????i???(????)??????c,???????????
• ????19