COMP8620 Lecture 8

1
COMP8620 Lecture 8

• Dynamic Programming

2
Dynamic Programming

• Principle of optimality
• If an intermediate state is visited in an optimal
  sequence of decisions, then the decisions leading
  to that state must also be optimal.

3
Bellmans equation

• to reach optimally state x in stage n1 it
  suffices to consider those policies that reach
  optimally each possible state u in previous stage
  n)
• (Bellman,1960)

4
Properties of DP

• Simple Subproblems
• We should be able to break the original problem
  to smaller sub-problems that have the same
  structure
• Optimal Substructure of the problems
• The optimal solution to the problem must be a
  composition of optimal solutions to subproblems
• Subproblem Overlap
• Optimal subproblems to unrelated problems can
  contain subproblems in common

5
Longest Common Subsequence (LCS)

• Problem Find the longest pattern of characters
  that is common to two text strings X and Y
• Subproblem find the LCS of prefixes of X and Y.
• Optimal substructure LCS of two prefixes is
  always a part of LCS of bigger strings
• A L G O R I T H M
• A L I G N M E N T

6
Longest Common Subsequence (LCS)

• Define Xi, Yj to be prefixes of X and Y of length
  i and j m X, n Y
• We store the length of LCS(Xi, Yj) in ci,j
• Trivial cases LCS(X0 , Yj ) and LCS(Xi, Y0) is
  empty (so c0,j ci,0 0 )
• Recursive formula for ci,j

cm,n is the final solution
7
Longest Common Subsequence (LCS)

• Need separate data structure to retrieve answer
• Algorithm runs in O(mn),
• cf Brute-force algorithm O(n 2m)

8
0-1 Knapsack problem

• Given a knapsack with maximum capacity W, and a
  set S consisting of n items
• Each item i has some weight wi and benefit value
  bi (all wi , bi and W are integer values)
• Problem How to pack the knapsack to achieve
  maximum total value of packed items?

9
0-1 Knapsack problem
Weight
Benefit value
bi
wi
Items
2
3
This is a knapsack Max weight W 20
4
3
5
4
8
5
9
10
10
0-1 Knapsack problem

• 0-1 problem each item is entirely accepted or
  rejected.

11
0-1 Knapsack problem Brute force

• n items 2n possible combinations of items.
• Go through all combinations, find the one with
  the most total value and with total weight less
  or equal to W
• Running time O(2n)

12
Dynamic Programming

• Consider DP
• What subproblem?

13
Defining a Subproblem

• Subproblem(?) find an optimal solution for Sk
  items labeled 1, 2, .. k within weight 20
• Either
• Use item k with one of the Si for i lt k or
• Dont use k and use the best Si so far

14
Defining a subproblem
k bk wk
1 2 3
2 3 4
3 4 5
4 5 8
5 9 10
k Sk Wk
1
2
3
4
5
15
Defining a subproblem
k bk wk
1 2 3
2 3 4
3 4 5
4 5 8
5 9 10
k Sk Wk
1 2 3
2
3
4
5
16
Defining a subproblem
k bk wk
1 2 3
2 3 4
3 4 5
4 5 8
5 9 10
k Sk Wk
1 2 3
2 5 7
3
4
5
17
Defining a subproblem
k bk wk
1 2 3
2 3 4
3 4 5
4 5 8
5 9 10
k Sk Wk
1 2 3
2 5 7
3 9 12
4
5
18
Defining a subproblem
k bk wk
1 2 3
2 3 4
3 4 5
4 5 8
5 9 10
k Sk Wk
1 2 3
2 5 7
3 9 12
4 14 20
5
19
Defining a subproblem
k bk wk
1 2 3
2 3 4
3 4 5
4 5 8
5 9 10
k Sk Wk
1 2 3
2 5 7
3 9 12
4 14 20
5
20
Defining a subproblem
k bk wk
1 2 3
2 3 4
3 4 5
4 5 8
5 9 10
k Sk Wk
1 2 3
2 5 7
3 9 12
4 14 20
5 14 17
But sol1,3,4,5 Sw 20 Sb 26!
21
What went wrong?

• Wrong subproblem
• The best way to pick k-1 items with a budget of
  20 does NOT necessarily chose the best way to
  choose the next item

22
Subproblem Mk II

• State includes how much weight I have used so far
• Consider filling the knapsack to weight w with
  up to k items
• Either I choose item k or not
• If I choose item k, I need to have filled my
  knapsack up to weight w-wk optimally, and then
  add item k, OR
• I fill my knapsack up to weigh w and skip item k

23
Recursive Formula for subproblems

• The best way to pack up to item k using at most
  weight w is either
• 1) the best subset of Sk-1 that has total weight
  w-wk plus item k or
• 2) the best subset of Sk-1 that has total weight
  w,

24
0-1 Knapsack Algorithm

• for w 0 to W
• B0,w 0
• for i 0 to n
• Bi,0 0
• for w 0 to W
• if wi lt w // item i can be part of the sol
• if bi Bi-1,w-wi gt Bi-1,w
• Bi,w bi Bi-1,w- wi // use i
• else
• Bi,w Bi-1,w // dont use i
• else
• Bi,w Bi-1,w // wi gt w

25
Running time

• for w 0 to W
• B0,w 0
• for i 0 to n
• Bi,0 0
• for w 0 to W
• lt the rest of the code gt

O(W)
Repeat n times
O(W)
Running time
O(nW)
Brute-force algorithm O(2n)
26
Example
Item wi bi
1 2 3
2 3 4
3 4 5
4 5 6
Data n 4 ( of elements) W 5 (max weight)
27
Example (2)
i
4
0
1
2
3
W
0
0
0
1
0
2
0
3
0
4
0
5
for w 0 to W B0,w 0
28
Example (3)
i
4
0
1
2
3
W
0
0
0
0
0
0
0
1
0
2
0
3
0
4
0
5
for i 0 to n Bi,0 0
29
Example (4)
Items 1 (2,3) 2 (3,4) 3 (4,5) 4 (5,6)
i
4
0
1
2
3
W
0
0
0
0
0
0
i1 bi3 wi2 w1 w-wi -1
0
1
0
0
2
0
3
0
4
0
5
if wi lt w // item i can be part of the
solution if bi Bi-1,w-wi gt Bi-1,w
Bi,w bi Bi-1,w- wi
else Bi,w Bi-1,w else
Bi,w Bi-1,w // wi gt w
30
Example (5)
Items 1 (2,3) 2 (3,4) 3 (4,5) 4 (5,6)
i
4
0
1
2
3
W
0
0
0
0
0
0
i1 bi3 wi2 w2 w-wi 0
0
1
0
0
2
3
0
3
0
4
0
5
if wi lt w // item i can be part of the
solution if bi Bi-1,w-wi gt Bi-1,w
Bi,w bi Bi-1,w- wi
else Bi,w Bi-1,w else
Bi,w Bi-1,w // wi gt w
31
Example (6)
Items 1 (2,3) 2 (3,4) 3 (4,5) 4 (5,6)
i
4
0
1
2
3
W
0
0
0
0
0
0
i1 bi3 wi2 w3 w-wi1
0
1
0
0
2
3
0
3
3
0
4
0
5
if wi lt w // item i can be part of the
solution if bi Bi-1,w-wi gt Bi-1,w
Bi,w bi Bi-1,w- wi
else Bi,w Bi-1,w else
Bi,w Bi-1,w // wi gt w
32
Example (7)
Items 1 (2,3) 2 (3,4) 3 (4,5) 4 (5,6)
i
4
0
1
2
3
W
0
0
0
0
0
0
i1 bi3 wi2 w4 w-wi2
0
1
0
0
2
3
0
3
3
0
4
3
0
5
if wi lt w // item i can be part of the
solution if bi Bi-1,w-wi gt Bi-1,w
Bi,w bi Bi-1,w- wi
else Bi,w Bi-1,w else
Bi,w Bi-1,w // wi gt w
33
Example (8)
Items 1 (2,3) 2 (3,4) 3 (4,5) 4 (5,6)
i
4
0
1
2
3
W
0
0
0
0
0
0
i1 bi3 wi2 w5 w-wi2
0
1
0
0
2
3
0
3
3
0
4
3
0
5
3
if wi lt w // item i can be part of the
solution if bi Bi-1,w-wi gt Bi-1,w
Bi,w bi Bi-1,w- wi
else Bi,w Bi-1,w else
Bi,w Bi-1,w // wi gt w
34
Example (9)
Items 1 (2,3) 2 (3,4) 3 (4,5) 4 (5,6)
i
4
0
1
2
3
W
0
0
0
0
0
0
i2 bi4 wi3 w1 w-wi-2
0
1
0
0
0
2
3
0
3
3
0
4
3
0
5
3
if wi lt w // item i can be part of the
solution if bi Bi-1,w-wi gt Bi-1,w
Bi,w bi Bi-1,w- wi
else Bi,w Bi-1,w else
Bi,w Bi-1,w // wi gt w
35
Example (10)
Items 1 (2,3) 2 (3,4) 3 (4,5) 4 (5,6)
i
4
0
1
2
3
W
0
0
0
0
0
0
i2 bi4 wi3 w2 w-wi-1
0
1
0
0
0
2
3
3
0
3
3
0
4
3
0
5
3
if wi lt w // item i can be part of the
solution if bi Bi-1,w-wi gt Bi-1,w
Bi,w bi Bi-1,w- wi
else Bi,w Bi-1,w else
Bi,w Bi-1,w // wi gt w
36
Example (11)
Items 1 (2,3) 2 (3,4) 3 (4,5) 4 (5,6)
i
4
0
1
2
3
W
0
0
0
0
0
0
i2 bi4 wi3 w3 w-wi0
0
1
0
0
0
2
3
3
0
3
3
4
0
4
3
0
5
3
if wi lt w // item i can be part of the
solution if bi Bi-1,w-wi gt Bi-1,w
Bi,w bi Bi-1,w- wi
else Bi,w Bi-1,w else
Bi,w Bi-1,w // wi gt w
37
Example (12)
Items 1 (2,3) 2 (3,4) 3 (4,5) 4 (5,6)
i
4
0
1
2
3
W
0
0
0
0
0
0
i2 bi4 wi3 w4 w-wi1
0
1
0
0
0
2
3
3
0
3
3
4
0
4
3
4
0
5
3
if wi lt w // item i can be part of the
solution if bi Bi-1,w-wi gt Bi-1,w
Bi,w bi Bi-1,w- wi
else Bi,w Bi-1,w else
Bi,w Bi-1,w // wi gt w
38
Example (13)
Items 1 (2,3) 2 (3,4) 3 (4,5) 4 (5,6)
i
4
0
1
2
3
W
0
0
0
0
0
0
i2 bi4 wi3 w5 w-wi2
0
1
0
0
0
2
3
3
0
3
3
4
0
4
3
4
0
5
3
7
if wi lt w // item i can be part of the
solution if bi Bi-1,w-wi gt Bi-1,w
Bi,w bi Bi-1,w- wi
else Bi,w Bi-1,w else
Bi,w Bi-1,w // wi gt w
39
Example (14)
Items 1 (2,3) 2 (3,4) 3 (4,5) 4 (5,6)
i
4
0
1
2
3
W
0
0
0
0
0
0
i3 bi5 wi4 w1..3
0
1
0
0
0
0
0
2
3
3
3
0
3
3
4
4
0
4
3
4
0
5
3
7
if wi lt w // item i can be part of the
solution if bi Bi-1,w-wi gt Bi-1,w
Bi,w bi Bi-1,w- wi
else Bi,w Bi-1,w else
Bi,w Bi-1,w // wi gt w
40
Example (15)
Items 1 (2,3) 2 (3,4) 3 (4,5) 4 (5,6)
i
4
0
1
2
3
W
0
0
0
0
0
0
i3 bi5 wi4 w4 w- wi0
0
1
0
0
0
0
0
2
3
3
3
0
3
4
4
3
0
4
4
5
3
0
5
7
3
if wi lt w // item i can be part of the
solution if bi Bi-1,w-wi gt Bi-1,w
Bi,w bi Bi-1,w- wi
else Bi,w Bi-1,w else
Bi,w Bi-1,w // wi gt w
41
Example (15)
Items 1 (2,3) 2 (3,4) 3 (4,5) 4 (5,6)
i
4
0
1
2
3
W
0
0
0
0
0
0
i3 bi5 wi4 w5 w- wi1
0
1
0
0
0
0
0
2
3
3
3
0
3
4
4
3
0
4
4
5
3
0
5
7
7
3
if wi lt w // item i can be part of the
solution if bi Bi-1,w-wi gt Bi-1,w
Bi,w bi Bi-1,w- wi
else Bi,w Bi-1,w else
Bi,w Bi-1,w // wi gt w
42
Example (16)
Items 1 (2,3) 2 (3,4) 3 (4,5) 4 (5,6)
i
4
0
1
2
3
W
0
0
0
0
0
0
i3 bi5 wi4 w1..3
0
1
0
0
0
0
0
0
2
3
3
3
3
0
3
4
4
4
3
0
4
4
5
3
0
5
7
7
3
if wi lt w // item i can be part of the
solution if bi Bi-1,w-wi gt Bi-1,w
Bi,w bi Bi-1,w- wi
else Bi,w Bi-1,w else
Bi,w Bi-1,w // wi gt w
43
Example (16)
Items 1 (2,3) 2 (3,4) 3 (4,5) 4 (5,6)
i
4
0
1
2
3
W
0
0
0
0
0
0
i3 bi5 wi4 w4
0
1
0
0
0
0
0
0
2
3
3
3
3
0
3
4
4
4
3
0
4
4
5
5
3
0
5
7
7
3
if wi lt w // item i can be part of the
solution if bi Bi-1,w-wi gt Bi-1,w
Bi,w bi Bi-1,w- wi
else Bi,w Bi-1,w else
Bi,w Bi-1,w // wi gt w
44
Example (17)
Items 1 (2,3) 2 (3,4) 3 (4,5) 4 (5,6)
i
4
0
1
2
3
W
0
0
0
0
0
0
i3 bi5 wi4 w5
0
1
0
0
0
0
0
0
2
3
3
3
3
0
3
4
4
4
3
0
4
4
5
5
3
0
5
7
7
7
3
if wi lt w // item i can be part of the
solution if bi Bi-1,w-wi gt Bi-1,w
Bi,w bi Bi-1,w- wi
else Bi,w Bi-1,w else
Bi,w Bi-1,w // wi gt w
45
Solution
i
4
0
1
2
3
W
0
0
0
0
0
0
0
1
0
0
0
0
0
0
2
3
3
3
3
0
3
4
4
4
3
0
4
4
5
5
3
0
5
7
7
7
3
46
Solution
i
4
0
1
2
3
W
0
0
0
0
0
0
0
1
0
0
0
0
0
0
2
3
3
3
3
0
3
4
4
4
3
0
4
4
5
5
3
0
5
7
7
7
3
47
Dynamic Programming

• Three basic components
• The recurrence relation (for defining the value
  of an optimal solution)
• The tabular computation (for computing the value
  of an optimal solution)
• The traceback (for delivering an optimal
  solution).
• Useful for solving certain types of problem

48
Exercise

• Do it yourself
• Find the edit distance between two strings
• Cost of adding a letter 1
• Cost of deleting a letter 1
• Cost of modifying a letter 1
• Cost of transposing 2 letters 1
• What is state?
• What is the recursion?
• Try on DYNAMIC ? DNAMCI

49

50
Idea

• Either
• Copy a letter (cost 0)
• Add a letter (cost 1)
• Delete a letter (cost 1)
• Modify a letter (cost 1)
• Transpose 2 letters
• S(i,j) edit cost of turning X0..i to Y0..j

51
Recursion
52

1. public static int editDist (String a, String b)
3. int n a.length() 1
4. int m b.length() 1
5. int dist new int nm
7. for (int i 0 i lt n i)
8. disti0 i
9. for (int j 0 j lt m j)
10. dist0j j

53

1. for (int i 1 i lt n i)
2. for (int j 1 j lt m j)
3. // Assume modify
4. int best disti-1j-1 1
5. if (a.charAt(i-1) b.charAt(j-1))
6. best disti-1j-1
7. if (disti-1j 1 lt best)
8. best disti-1j 1 // Add
9. if (distij-1 1 lt best)
10. best distij-1 1 //
    Delete
11. if (
12. i gt 1 j gt 1
13. a.charAt(i-1) b.charAt(j-2)
14. a.charAt(i-2) b.charAt(j-1)
15. disti-2j-2 1 lt best
16. )
17. best disti-2j-2 1 //
    Transpose
18. distij best

54
Running it

• java -cp . EditDist dynamic dnamci
• 0 1 2 3 4 5 6
• 1 0 1 2 3 4 5
• 2 1 1 2 3 4 5
• 3 2 1 2 3 4 5
• 4 3 2 1 2 3 4
• 5 4 3 2 1 2 3
• 6 5 4 3 2 2 2
• 7 6 5 4 3 2 2
• dynamic dnamci 2

55
References

• Short section (Section 2.4) in Search
  Methodologies, E.K. Burke and G Kendall (Eds),
  Springer 2005
• In depth in Combinatorial Data Analysis
  Optimization by Dynamic Programming, L.J.
  Hubert, P Arable and J. Meulman, SIAM, 2001
  (electronic book)

56

• Next weekConstraint Programming (Jason Li)