Chapter 16 Greedy Algorithms

1
Chapter 16 Greedy Algorithms
2
Introduction

• Optimization problem there can be many possible
  solution. Each solution has a value, and we wish
  to find a solution with the optimal (minimum or
  maximum) value
• Greedy algorithm an algorithmic technique to
  solve optimization problems
• Always makes the choice that looks best at the
  moment.
• Makes a locally optimal choice in the hope that
  this choice will lead to a globally optimal
  solution

3
An Activity-Selection Problem
4
Overview

• Scheduling a resource among several competing
  activities
• Definition
• S1, 2,, n activities that wish to use a
  resource
• Each activity has a start time si and a finish
  time fi, where si ? fi
• If selected, activity i takes place during the
  half-open interval si, fi)
• Activities i and j are compatible if si, fi) and
  sj, fj) dont overlap
• si ? fj or sj ? fi
• The activity-selection problem is to select a
  maximum-size set of mutually compatible
  activities
• Greedy-Activity-Selector Algorithm
• Suppose f1 ? f2 ? f3 ? ? fn
• ?(n) if the activities are already sorted
  initially by their finish times

5
Greedy-Activity-Selector
fj is always the max finishing time of any
activity in Afj max fk k ?A
// Selected activities
// The most recent addition to A
6
(No Transcript)
7
Greedy-Activity-Selector (Cont.)

• The activity picked next is always the one with
  the earliest finish time that can be legally
  scheduled
• Greedy in the sense that it leaves as much
  opportunity as possible for the remaining
  activities to be scheduled
• The greedy choice is the one that maximizes the
  amount of unscheduled time remaining

8
Proving The Greedy Algorithm Correct

• Show there is an optimal solution that begins
  with a greedy choice (with activity 1, which as
  the earliest finish time)
• Suppose A ? S in an optimal solution
• Order the activities in A by finish time. The
  first activity in A is k
• If k 1, the schedule A begins with a greedy
  choice
• If k ? 1, show that there is an optimal solution
  B to S that begins with the greedy choice,
  activity 1
• Let B A k ? 1
• f1 ? fk ? activities in B are disjoint
  (compatible)
• B has the same number of activities as A
• ? B is optimal

9
Proving The Greedy Algorithm Correct (Cont.)

• Once the greedy choice of activity 1 is made, the
  problem reduces to finding an optimal solution
  for the activity-selection problem over those
  activities in S that are compatible with activity
  1
• Optimal Substructure
• If A is optimal to S, then A A 1 is
  optimal to Si ?S si ? f1
• Why?
• If we could find a solution B to S with more
  activities than A, adding activity 1 to B would
  yield a solution B to S with more activities than
  A ? contradicting the optimality of A
• After each greedy choice is made, we are left
  with an optimization problem of the same form as
  the original problem
• By induction on the number of choices made,
  making the greedy choice at every step produces
  an optimal solution

10
Elements of the Greedy Strategy
11
Overview

• An greedy algorithm makes a sequence of choices,
  each of the choices that seems best at the moment
  is chosen
• NOT always produce an optimal solution
• Two ingredients that are exhibited by most
  problems that lend themselves to a greedy
  strategy
• Greedy-choice property
• Optimal substructure

12
Greedy-Choice Property

• A globally optimal solution can be arrived at by
  making a locally optimal (greedy) choice
• Make whatever choice seems best at the moment and
  then solve the sub-problem arising after the
  choice is made
• The choice made by a greedy algorithm may depend
  on choices so far, but it cannot depend on any
  future choices or on the solutions to
  sub-problems
• Usually progress in a top-down fashion ? making
  one greedy choice on after another, iteratively
  reducing each given problem instance to a smaller
  one
• Of course, we must prove that a greedy choice at
  each step yields a globally optimal solution
• Theorem 17.1 ? transformation

13
Optimal Substructure

• A problem exhibits optimal substructure if an
  optimal solution to the problem contains within
  it optimal solutions to sub-problems
• If an optimal solution A to S begins with
  activity 1, then A A 1 is optimal to Si
  ?S si ? f1
• Theorem 17.1 ?induction

14
Knapsack Problems

• You want to pack n items in your luggage
• The ith item is worth vi dollars and weighs wi
  pounds
• Take a valuable a load as possible, but cannot
  exceed W pounds
• vi , wi, W are integers
• 0-1 knapsack ? each item is taken or not taken
• Fractional knapsack ? fractions of items can be
  taken
• Both exhibit the optimal-substructure property
• 0-1 consider a optimal solution. If item j is
  removed from the load, the remaining load must be
  the most valuable load weighing at most W-wj
• Fractional If w of item j is removed from the
  optimal load, the remaining load must be the most
  valuable load weighing at most W-w that can be
  taken from other n-1 items plus wj w of item j

15
Greedy or Not?

• Fractional knapsack can be solvable by the greedy
  strategy
• Compute the value per pound vi/wi for each item
• Obeying a greedy strategy, we take as much as
  possible of the item with the greatest value per
  pound.
• If the supply of that item is exhausted and we
  can still carry more, we take as much as possible
  of the item with the next value per pound, and so
  forth until we cannot carry any more
• O(n lg n) (we need to sort the items by value per
  pound)
• Algorithm?
• Correctness?

16
(No Transcript)
17
Greedy or Not? (Cont.)

• 0-1 knapsack cannot be solved by the greedy
  strategy
• Unable to fill the knapsack to capacity, and the
  empty space lowers the effective value per pound
  of the load
• We must compare the solution to the sub-problem
  in which the item is included with the solution
  to the sub-problem in which the item is excluded
  before we can make the choice
• Dynamic Programming

18
Huffman Codes
19
Overview

• Huffman codes compressing data (savings of 20
  to 90)
• Huffmans greedy algorithm uses a table of the
  frequencies of occurrence of each character to
  build up an optimal way of representing each
  character as a binary string

20
Prefix Code

• Prefix(-free) code no codeword is also a prefix
  of some other codewords (Un-ambiguous)
• An optimal data compression achievable by a
  character code can always be achieved with a
  prefix code
• Simplify the encoding (compression) and decoding
• Encoding abc ? 0 . 101. 100 0101100
• Decoding 001011101 0. 0. 101. 1101 ? aabe
• Use binary tree to represent prefix codes for
  easy decoding
• An optimal code is always represented by a full
  binary tree, in which every non-leaf node has two
  children
• C leaves and C-1 internal nodes (Exercise
  B.5-3)
• Cost

21
(Not optimal)
22
Constructing A Huffman Code

• C is a set of n characters
• Each character c ? C is an object with a
  frequency, denoted by fc
• The algorithm builds the tree T in a bottom-up
  manner
• Begin with C leaves and perform a sequence of
  C-1 merging
• A min-priority queue Q, keyed on f, is used to
  identify the two least-frequent objects to merge
  together
• The result of the merger of two objects is a new
  object whose frequency is the sum of the
  frequencies of the two objects that were merged

23
(No Transcript)
24
Constructing A Huffman Code (Cont.)
Total computation time O(n lg n)
O(lg n)
O(lg n)
O(lg n)
O(lg n)
25
Lemma 16.2 Greedy-Choice

• Let C be an alphabet in which each character c ?
  C has a frequency fC. Let x and y be two
  characters in C having the lowest frequencies.
  Then there exists an optimal prefix code for C in
  which the codewords for x and y have the same
  length and differ only in the last bit

Since each swap does not increase the cost, the
resulting tree T is also anoptimal tree
26
Proof of Lemma 16.2

• Without loss of generality, assume fa?fb and
  fx?fy
• The cost difference between T and T is

B(T) ? B(T), but T is optimal, B(T)?B(T)?
B(T) B(T)Therefore T is an optimal tree in
which x and y appear as sibling leaves of maximum
depth
27
Greedy-Choice?

• Define the cost of a single merger as being the
  sum of the frequencies of the two items being
  merged
• Of all possible mergers at each step, HUFFMAN
  chooses the one that incurs the least cost

28
Lemma 16.3 Optimal Substructure

• Let C C x, y ? z
• fz fx fy
• Let T be any tree representing an optimal prefix
  code for C Then the tree T, obtained from T by
  replacing the leaf node for z with an internal
  node having x and y as children, represent an
  optimal prefix code for C
• Observation B(T) B(T) fx fy ? B(T)
  B(T)-fx-fy
• For each c ?C x, y ? dT(c) dT(c)?
  fcdT(c) fcdT(c)
• dT(x) dT(y) dT(z) 1
• fxdT(x) fydT(y) (fx fy)(dT(z)
  1) fzdT(z) (fx fy)

29
B(T) B(T)-fx-fy
30
Proof of Lemma 16.3

• Prove by contradiction.
• Suppose that T does not represent an optimal
  prefix code for C. Then there exists a tree T
  such that B(T) lt B(T).
• Without loss of generality, by Lemma 16.2, T
  has x and y as siblings. Let T be the tree T
  with the common parent x and y replaced by a leaf
  with frequency fz fx fy. Then
• B(T) B(T) - fx fy lt B(T) fx
  fy B(T)
• T is better than T ? contradiction to the
  assumption that T is an optimal prefix code for
  C

31
Self Study

• Section 16.4 Theoretical foundations for greedy
  methods
• The Matroid theory
• Note that the matriod theory does not cover all
  cases for which a greedy method applies
• ex. activity-selection problem and Huffman coding
  problem
• Section 16.5 A task-scheduling problem
• Can be solved using matroids
• PP. 371 376
• Solve the activity-selection problem from the
  viewpoint of dynamic programming (Chapter 15).