Lecture

1
CS 312 Algorithm Analysis

• Lecture 20 (Divisible) Knapsack

2
Announcements

• Project 2
• Improvement due today
• Remember Updated hints on Blackboard
• Questions?
• A word on cooperation
• Project 3
• Help session on Tues at 1pm with David Manning
• Help session on Tues at 5pm with Michael
  Deardeuff
• Mid-term
• Next Thu-Sat (3/1-3/3)
• In the Testing Center
• Check closing times!
• Coverage up through Greedy algorithms
• HW keys available on Blackboard
• Study list coming this weekend

3
Objectives

• Lecture 20 (Divisible) Knapsack
• More Greedy Algorithms
• Remember the general form of a greedy algorithm
• Define the (Divisible) Knapsack Problem
• Learn how to formulate the problem in algebraic
  terms
• See the formulation as a linear program (for
  future reference)
• Work through an example

• Lecture 19 Prims Algorithm and Huffman
  Encoding
• More Greedy Algorithms
• Prims Algorithm for MST
• Huffman Encoding

4
Generic Greedy Alg.

• function greedy (C)
• Input set of candidates C
• Output solution S (a set)
• S ? ?
• while (C ! ? !solution(S))
• x ? select(C)
• C ? C \ x
• if feasible(S?x) then
• S ? S ? x
• if solution(S) then return S
• else there are no solutions

5
The Knapsack Problem

• Given n objects with weight wi and value vi
• Knapsack can only hold a total weight of W
• Fill Knapsack
• Maximize total value
• Dont exceed W

6
The Knapsack Problem
Mathematically, represent a selection from n
objects by a vector x
Fix Order of Objects
7
The Knapsack Problem
Mathematically, represent a selection from n
objects by a vector x
Fix Order of Objects
8
The Knapsack Problem

• The problem is written mathematically as

where vigt0, wigt0, and 0ltxilt1 for all 1lt iltn.
9
The Knapsack Problem

• The problem is written mathematically as

where vigt0, wigt0, and 0ltxilt1 for all 1lt iltn.
10
The Knapsack Problem

• The problem is written mathematically as

where vigt0, wigt0, and 0ltxilt1 for all 1lt iltn.
11
The Knapsack Problem

• Problems of this form are called Linear Programs.

12
The Knapsack Problem

• Problems of this form are called Linear Programs.

They are a special case of general mathematical
programs, or optimization problems.
13
The Knapsack Problem

• Problems of this form are called Linear Programs.

If we can introduce a meaningful Selection
Function, we can design Greedy Algorithms to try
to solve the optimization problem!
14
Knapsack
function knapsack (w1n, v1n, W) Input
vector w of weights, vector v of values, knapsack
capacity W Output vector x of object counts for
i 1 to n do xi ? 0 weight ? 0 while weight lt
W do i ? the best remaining object if
weight wi lt W then xi ? 1
weight ? weight wi else xi
? (W - weight)/wi weight ? W return x
15
Knapsack Selection Fn.

• What is the selection function if
• want to maximize profit?
• want to minimize profit (cost)?

16
The Knapsack Problem

• Greedy Algorithm function knapsack(w1..n,v1..n
  ,W)array 1..n
• initialization
• for i 1 to n do xi ? 0
• weight ? 0
• Greedy Loop
• while weight lt W do
• i ? the best remaining object
• if weightwiltW then
• xi ? 1
• weight ? weight wi
• else xi ? (W weight)/wi
• weight ? W
• return x

17
The Knapsack Problem

• Greedy Algorithm function knapsack(w1..n,v1..n
  ,W)array 1..n
• initialization
• for i 1 to n do xi ? 0
• weight ? 0
• Greedy Loop
• while weight lt W do
• i ? the best remaining object
• if weightwiltW then
• xi ? 1
• weight ? weight wi
• else xi ? (W weight)/wi
• weight ? W
• return x

• Selection Function Ideas
• Most valuable
• Lightest weight
• Highest value/unit weight

18
The Knapsack Problem

• Example Let W100 and consider

• Selection Function Ideas
• Most valuable
• Lightest weight
• Highest value/unit weight

19
The Knapsack Problem

• Example Let W100 and consider

• Selection Function Ideas
• Most valuable
• Lightest weight
• Highest value/unit weight

weight 0
20
The Knapsack Problem

• Example Let W100 and consider

• Selection Function Ideas
• Most valuable
• Lightest weight
• Highest value/unit weight

weight 0
21
The Knapsack Problem

• Example Let W100 and consider

• Selection Function Ideas
• Most valuable
• Lightest weight
• Highest value/unit weight

weight 0
22
The Knapsack Problem

• Example Let W100 and consider

• Selection Function Ideas
• Most valuable
• Lightest weight
• Highest value/unit weight

weight 30
23
The Knapsack Problem

• Example Let W100 and consider

• Selection Function Ideas
• Most valuable
• Lightest weight
• Highest value/unit weight

weight 30
24
The Knapsack Problem

• Example Let W100 and consider

• Selection Function Ideas
• Most valuable
• Lightest weight
• Highest value/unit weight

weight 30
25
The Knapsack Problem

• Example Let W100 and consider

• Selection Function Ideas
• Most valuable
• Lightest weight
• Highest value/unit weight

weight 80
26
The Knapsack Problem

• Example Let W100 and consider

• Selection Function Ideas
• Most valuable
• Lightest weight
• Highest value/unit weight

weight 80
27
The Knapsack Problem

• Example Let W100 and consider

• Selection Function Ideas
• Most valuable
• Lightest weight
• Highest value/unit weight

weight 80
28
The Knapsack Problem

• Example Let W100 and consider

• Selection Function Ideas
• Most valuable
• Lightest weight
• Highest value/unit weight

weight 100
29
The Knapsack Problem

• Example Let W100 and consider

• Selection Function Ideas
• Most valuable
• Lightest weight
• Highest value/unit weight

Score 1 146
weight 100
30
The Knapsack Problem

• Example Let W100 and consider

• Selection Function Ideas
• Most valuable
• Lightest weight
• Highest value/unit weight

Score 1 146
weight 0
31
The Knapsack Problem

• Example Let W100 and consider

• Selection Function Ideas
• Most valuable
• Lightest weight
• Highest value/unit weight

Score 1 146
weight 100
32
The Knapsack Problem

• Example Let W100 and consider

• Selection Function Ideas
• Most valuable
• Lightest weight
• Highest value/unit weight

Score 1 146 Score 2 156
weight 100
33
The Knapsack Problem

• Example Let W100 and consider

• Selection Function Ideas
• Most valuable
• Lightest weight
• Highest value/unit weight

Score 1 146 Score 2 156
weight 0
34
The Knapsack Problem

• Example Let W100 and consider

• Selection Function Ideas
• Most valuable
• Lightest weight
• Highest value/unit weight

Score 1 146 Score 2 156
weight 0
35
The Knapsack Problem

• Example Let W100 and consider

• Selection Function Ideas
• Most valuable
• Lightest weight
• Highest value/unit weight

Score 1 146 Score 2 156
weight 0
36
The Knapsack Problem

• Example Let W100 and consider

• Selection Function Ideas
• Most valuable
• Lightest weight
• Highest value/unit weight

Score 1 146 Score 2 156
weight 30
37
The Knapsack Problem

• Example Let W100 and consider

• Selection Function Ideas
• Most valuable
• Lightest weight
• Highest value/unit weight

Score 1 146 Score 2 156
weight 30
38
The Knapsack Problem

• Example Let W100 and consider

• Selection Function Ideas
• Most valuable
• Lightest weight
• Highest value/unit weight

Score 1 146 Score 2 156
weight 40
39
The Knapsack Problem

• Example Let W100 and consider

• Selection Function Ideas
• Most valuable
• Lightest weight
• Highest value/unit weight

Score 1 146 Score 2 156
weight 40
40
The Knapsack Problem

• Example Let W100 and consider

• Selection Function Ideas
• Most valuable
• Lightest weight
• Highest value/unit weight

Score 1 146 Score 2 156
weight 60
41
The Knapsack Problem

• Example Let W100 and consider

• Selection Function Ideas
• Most valuable
• Lightest weight
• Highest value/unit weight

Score 1 146 Score 2 156
weight 60
42
The Knapsack Problem

• Example Let W100 and consider

• Selection Function Ideas
• Most valuable
• Lightest weight
• Highest value/unit weight

Score 1 146 Score 2 156
weight 100
43
The Knapsack Problem

• Example Let W100 and consider

• Selection Function Ideas
• Most valuable
• Lightest weight
• Highest value/unit weight

Score 1 146 Score 2 156 Score 3 164
weight 100
44
The Knapsack Problem

• Theorem If objects are selected in order of
  decreasing vi/wi, then algorithm knapsack finds
  an optimal solution.

Efficiency the Algorithm requires time that
grows in O(nlogn) IF we keep objects in a
priority queue, implemented as a binary heap.

• Selection Function Ideas
• Most valuable
• Lightest weight
• Highest value/unit weight

Score 1 146 Score 2 156 Score 3 164
45
Assignment

• HW 15 See the link on the schedule.
• Question 1. (Divisible) Knapsack. Consider the
  following instance of the knapsack problem, where
  the number of objects n4, and the capacity W80.
• Solve this problem using the greedy algorithm of
  we discussed in class. What is the value of the
  optimal load? What are the values of the
  selection vector x?
• Question 2. Identify the elements of a greedy
  algorithm in the pseudo-code for Kruskals
  algorithm.
• Reading Sections 6.1-6.3 on Dynamic Programming