ESI 6448 Discrete Optimization Theory

1
ESI 6448Discrete Optimization Theory

• Section number 5643
• Lecture 7

2
Last class

• Nonbipartite matching
• blossom makes it difficult
• detect / shrink blossom and find augmenting path
• Dynamic programming
• principle of optimality
• examples shortest path problem, ULS

3
Nonbipartite matching

• A matching M is optimal iff there is no
  augmenting path w.r.t. M
• due to blossom (cycle w/ 2k1 nodes and k matched
  edges), we cant use bipartite algorithm
• blossom has 2 alternating paths from its basis to
  a node in it

9
10
8
7
6
7
9
5
3
1
0
w
v
0
v
3
2
1
4
5
6
4
2
8
10
4
Shrinking a blossom

• replacing a blossom b with a single node
• G (V, E) ? G/b (V/b, E/b)
• Suppose that while searching for an augmenting
  path from an exposed node u in G w.r.t. M, we
  discovered a blossom b, then
• i) there is an alternating path from u to any
  node of b ending with a matched edge.
• ii) there is an augmenting path from u in G
  w.r.t. M iff there is one from u in G/b w.r.t.
  M/b.

5
Algorithm

• Start from an empty matching and repeatedly
  choose an exposed vertex u to find an augmenting
  path from it
• If a blossom is found, shrink it and continue
  search
• If no augmenting path is found, u will never be
  chosen again, choose a different exposed node
• If augmenting path is found, reconstruct blossom
  and backtrack the path and augment the matching

6
Dynamic programming

• Basic idea
• Principle of optimality piece of optimal
  solutions are themselves optimal
• To find an optimal solution, first find optimal
  subsolution (define recurrence)
• Properties
• Principle of optimality
• overlapping subproblems
• top-down by recursion
• bottom-up by table

7
Example

• Shortest path problem
• d(v) length of shortest path from s to v,
  thend(v) mini ?V-(v) d(i) civ
• recurrence
• Dk(i) length of a shortest path from s to i
  containing at most k arcs
• Dk(i) min Dk-1(i), mini ?V-(v) d(i) civ

s
t
p
8
Example
j
4
2
7
a
d
g
5
2
5
5
2
k
1
1
3
6
3
1
s
b
e
h
t
2
4
3
4
l
4
2
4
1
2
2
c
f
i
3
m
9
Optimal subtree

• Given a tree T (V, E) w/ root r in V and
  weights cv for v in V, find a subtree of T rooted
  at r of maximum positive weight.

-2
1
2
-8
2
3
-1
-1
-6
-1
2
4
5
6
7
8
5
3
9
10
11
12
13
-2
3
3
10
What we have

• For each node v,
• predecessor p(v) on the unique path from r to v
• for v ? r, set of immediate successorS(v) w ?
  V p(w) v
• Let T(v) be the subtree of T rooted at v
  containing all nodes w s.t. path from r to w
  contains v

-2
1
2
-8
2
3
-1
-1
-6
-1
2
4
5
6
7
8
5
3
9
10
11
12
13
-2
3
3
11
Recurrence

• subproblems
• H(v) optimal solution value of subtree problem
  on T(v)
• If there is an optimal subtree of T(v),
• it must contain v
• it may also contain subtrees of T(w) rooted at w
  for w?S(v)
• recurrence H(v) max 0, cv ?w ?S(v) H(w)
• starting from leavesv ? V is a leaf, then H(v)
  max0, cv

12
Algorithm

• Starting from leaves, scan nodes towards the root
  r, removing all subtrees T(u) w/ H(u) 0
• O(n)

-2
1
2
-8
2
3
-1
-1
-6
-1
2
4
5
6
7
8
5
3
9
10
11
12
13
-2
3
3
13
Knapsack problem

• Shortest problem, ULS, optimal subtree
• Efficient Optimization Property
• O(n), O(n2), O(n)
• Knapsack problems are in general more difficult.
• DP for knapsack is efficient if the size of data
  is restricted
• 0-1 knapsack
• integer knapsack

14
0-1 knapsack

• z max ?nj1 cjxj ?nj1ajxj ? b
  x ? Bn, ajs and b are positive
• Think about a subproblem Pr(?)zr(?) max ?rj1
  cjxj ?rj1ajxj ? ?
  x ? Br
• relationship among subsolutions?

15
Recurrence

• Let x be an optimal solution
• for xr
• if xr 0, fr(?) fr-1(?)
• if xr 1, fr(?) cr fr-1(? ar)
• fr(?) max fr-1(?), cr fr-1(? ar)
• starting from r 0f0(?) 0 for ? ? 0
• for backtracking optimal solution, keep an
  indicatorpr(?) 0 if fr(?) fr-1(?), 1 o.w.

16
Algorithm

• Starting from r 0 and ? 0, increase ? by 1
  and compute fr(?) for each r until r n and ?
  b
• backtracking
• From r n and ? b, decrease r by 1 and look at
  the value of pr(?)
• pr(?) 0, then xr 0 and r r 1 and ? ?
• pr(?) 1, then xr 1 and r r 1 and ? ?
  ar
• O(nb)

17
Example

• z max 10x1 7x2 25x3 24x4
  2x1 x2 6x3 5x4 ? 7 x ?
  B4, fr(?) max fr-1(?), cr fr-1(? ar)

18
Integer knapsack

• z max ?nj1 cjxj ?nj1ajxj ? b
  x ? Zn, ajs and b are positive
• mimic 0-1 casesubproblem Pr(?) gr(?) max
  ?rj1 cjxj ?rj1ajxj ? ?
  x ? Zr

19
Recurrence

• Let x be an optimal solution
• for xr
• if xr 0, gr(?) gr-1(?)
• if xr t gt 0, gr(?) crt gr-1(? tar)
• gr(?) max0 ? t ? ??/ar? crt gr-1(? tar)
• too many (??/ar? 1) subsolutions to compute
• if xr t ? 1, modified solution w/ xr 1 is
  optimal for Pr(? ar)hence gr(?) cr gr(?
  ar) for x gt 0
• gr(?) max gr-1(?), cr gr(? ar)

20
Algorithm

• Starting from r 0 and ? 0, increase ? by 1
  and compute fr(?) for each r until r n and ?
  b
• backtracking
• From r n and ? b, decrease r by 1 and look at
  the value of pr(?)
• pr(?) 0, then xr 0 and r r 1 and ? ?
• while pr(?) 1, xr and r r and ? ? ar
• O(nb)

21
Example

• z max 7x1 9x2 2x3 15x4 3x1
  4x2 x3 7x4 ? 10 x ? Z4,
  gr(?) max gr-1(?), cr gr(? ar)

22
Alternative ways

• Just consider gn(?) ( h(?))
• recurrence h(?) max 0, maxj aj?? cj h(?
  aj) from xj ? 1 ? h(?)
  cj h(? aj)
• z max 7x1 9x2 2x3 15x4 3x1
  4x2 x3 7x4 ? 10 x ? Z4
• h(0) 0h(1) max 0, 0, 0,
  02, 0 2h(2) max 0, 0,
  0, 22, 0 4h(3) max 0, 07,
  0, 42, 0 7h(4) max 0, 07,
  09, 72, 0 9h(5) max 0,
  47, 29, 92, 0 11h(6) max 0,
  77, 49, 112, 0 14h(7) max
  0, 97, 79, 142, 015 16h(8) max
  0, 117, 99, 162, 215 18 h(9) max
  0, 147, 119, 182, 415 21h(10) max 0,
  167, 149, 212, 715 23

23
Alternative ways

• Longest path problem
• D (V, A), V 0, ..., b,E (?, ?aj) 0 ?
  ? aj ? b, 1 ? j ? n, weight of cj ? (?, ?1)
  0 ? ? 1 ? b, weight of 0
• h(?) length of longest path from 0 to ?
• z max 7x1 9x2 2x3 15x4 3x1
  4x2 x3 7x4 ? 10 x ? Z4

7
7
7
7
7
7
7
7
2
2
2
2
2
2
2
2
2
2
0
1
2
3
4
5
6
7
8
9
10
9
9
9
9
9
9
9
15
15
15
15
24
DP and greedy algorithm

• An optimal solution contains
• optimal subsolution
• subsolution w/ greedy choice
• How to check optimality of solution?
• DP show that principle of optimality holds
• greedy algorithm show the above property
  orprovide a matroid for the problem
• subset system (N, F) is a matroid iff
• i) N is a finite, nonempty set.
• ii) If B ? F and A ? B then A ? F.
• iii) If A ? F and B ? F and A lt B then there
  is x ? B A s.t. A ? x ? F.

25
Today

• Dynamic programming
• optimal subtree of a tree
• knapsack problems
• 0-1 knapsack
• integer knapsack
• comparison w/ greedy algorithm