Greedy Algorithms - PowerPoint PPT Presentation

About This Presentation
Title:

Greedy Algorithms

Description:

Greedy Algorithms Dr. Yingwu Zhu – PowerPoint PPT presentation

Number of Views:382
Avg rating:3.0/5.0
Slides: 40
Provided by: zhu131
Category:

less

Transcript and Presenter's Notes

Title: Greedy Algorithms


1
Greedy Algorithms
  • Dr. Yingwu Zhu

2
Greedy Technique
  • Constructs a solution to an optimization problem
    piece by
  • piece through a sequence of choices that are
  • Feasible satisfying the prob. constraints
  • locally optimal the best local choice
  • Irrevocable cannot be changed on subsequent
    steps once made
  • For some problems, yields an optimal solution for
    every instance.
  • For most, does not but can be useful for fast
    approximations.

3
DP vs. Greedy Algorithms
  • Both to solve problems exhibiting optimal
    substructure
  • DP
  • Solution to the problem assembled from the
    solutions to subproblems by considering various
    choices
  • Greedy algroithms
  • Greedy-choice property the locally optimal
    choice is made w/o considering results from
    subproblems
  • However, DP could be overkill sometimes

4
Applications of the Greedy Strategy
  • Optimal solutions
  • change making for normal coin denominations
  • minimum spanning tree (MST)
  • single-source shortest paths (Dijkstras
    algorithm)
  • simple scheduling problems
  • Huffman codes
  • Approximations
  • traveling salesman problem (TSP)
  • knapsack problem
  • other combinatorial optimization problems

5
An Activity-Selection Problem
  • Suppose A set of activities Sa1, a2,, an
  • They use resources, such as lecture hall, one
    lecture at a time
  • Each ai, has a start time si, and finish time fi,
    with 0? silt filt?.
  • ai and aj are compatible if si, fi) and sj, fj)
    do not overlap
  • Goal select maximum-size subset of mutually
    compatible activities.
  • Start from dynamic programming, then greedy
    algorithm, see the relation between the two.

6
Activity-Selection Problem
  • Problem get your moneys worth out of a carnival
  • Buy a wristband that lets you onto any ride
  • Lots of rides, each starting and ending at
    different times
  • Your goal ride as many rides as possible
  • Another, alternative goal that we dont solve
    here maximize time spent on rides
  • Welcome to the activity selection problem

7
Activity-Selection
  • Formally
  • Given a set S of n activities S a1, a2,, an
  • si start time of activity ai
  • fi finish time of activity ai
  • Find max-size subset A of compatible activities
  • Assume (wlog) that f1 ? f2 ? ? fn

8
DP Solution
  • Optimal substructure?

9
DP solution step 1
  • Optimal substructure of activity-selection
    problem.
  • Assume that f1 ? ?fn (otherwise, sort them by
    fi)
  • Define Sijak fi? skltfk?sj, i.e., all
    activities starting after ai finished and ending
    before aj begins.
  • Define two fictitious activities a0 with f00 and
    an1 with sn1?
  • So f0 ?f1 ? ?fn1.
  • Then an optimal solution including ak to Sij
    contains within it the optimal solution to Sik
    and Skj.

10
DP solution step 2
  • A recursive solution
  • Let ci,j be of activities in a maximum-size
    subset of mutually compatible activities in Sij.
    So the solution is c0,n1S0,n1.
  • Ci,j 0 if Sij?

    maxci,kck,j1 if Sij??
  • iltkltj and ak?Sij

11
Greedy Algorithms
  • Greedy choice
  • Intuition we should choose an activity that
    leaves the resource available for as many other
    activities as possible
  • So, consider the locally optimal choice
  • Select the activity ak with the earliest finish
    time in Si,j
  • Unlike DP solution, after the local greedy
    choice, only one subproblem remains!
  • One big question
  • Is our intuition correct?
  • We have to prove it is safe to make the greedy
    choice

12
Justify Greedy Choice
  • Theorem 16.1 consider any nonempty subproblem
    Sij, and let am be the activity in Sij with
    earliest finish time fmminfk ak ? Sij, then
  • Activity am is used in some maximum-size subset
    of mutually compatible activities of Sij.
  • The subproblem Sim is empty, so that choosing am
    leaves Smj as the only one that may be nonempty.
  • Proof of the theorem (p418)

13
Top-Down Rather Than Bottom-Up
  • To solve Sij, choose am in Sij with the earliest
    finish time, then solve Smj, (Sim is empty)
  • It is certain that optimal solution to Smj is in
    optimal solution to Sij.
  • No need to solve Smj ahead of Sij.
  • Subproblem pattern Si,n1.

14
Recursive Solution
recursive_select(s, f, k, n) m k1
while (m lt n sm lt fk) m if (m lt
n) return am U recursive_select(s, f, m, n)
else return Ø
15
Optimal Solution Properties
  • In DP, optimal solution depends
  • How many subproblems to divide. (2 subproblems)
  • How many choices to determine which subproblem to
    use. (j-i-1 choices)
  • However, the above theorem (16.1) reduces both
    significantly
  • One subproblem (the other is sure to be empty).
  • One choice, i.e., the one with earliest finish
    time in Sij.
  • Moreover, top-down solving, rather than bottom-up
    in DP.
  • Pattern to the subproblems that we solve, Sm,n1
    from Sij.
  • Pattern to the activities that we choose. The
    activity with earliest finish time.
  • With this local optimal, it is in fact the global
    optimal.

16
Elements of greedy strategy
  • Determine the optimal substructure
  • Develop the recursive solution
  • Prove one of the optimal choices is the greedy
    choice yet safe
  • Show that all but one of subproblems are empty
    after greedy choice
  • Develop a recursive algorithm that implements the
    greedy strategy
  • Convert the recursive algorithm to an iterative
    one.

17
Change-Making Problem
  • Given unlimited amounts of coins of denominations
    d1 gt gt dm ,
  • give change for amount n with the least number of
    coins
  • Example d1 25c, d2 10c, d3 5c, d4 1c
    and n 48c
  • Greedy solution
  • Greedy solution is
  • optimal for any amount and normal set of
    denominations
  • may not be optimal for arbitrary coin
    denominations
  • (4,3,1) for 6

18
Minimum Spanning Tree (MST), p623-628
  • Spanning tree of a connected graph G a connected
    acyclic subgraph of G that includes all of Gs
    vertices
  • Minimum spanning tree of a weighted, connected
    graph G a spanning tree of G of minimum total
    weight
  • Example

6
c
a
1
4
2
d
b
3
19
Prims MST algorithm (p634-636)
  • Start with tree T1 consisting of one (any) vertex
    and grow tree one vertex at a time to produce
    MST through a series of expanding subtrees T1,
    T2, , Tn
  • On each iteration, construct Ti1 from Ti by
    adding vertex not in Ti that is closest to those
    already in Ti (this is a greedy step!)
  • Stop when all vertices are included

20
Prims MST algorithm
  • Start with tree T1 consisting of one (any) vertex
    and grow tree one vertex at a time to produce
    MST through a series of expanding subtrees T1,
    T2, , Tn
  • On each iteration, construct Ti1 from Ti by
    adding vertex not in Ti that is closest to those
    already in Ti (this is a greedy step!)
  • Stop when all vertices are included

21
Prims algorithm
Step 0 Original graph
Step 1 D is chose as an arbitrary starting node
Step 3 F is added into the MST
Step 2 A is added into the MST
22
Prims algorithm
Step 5 E is added into the MST
Step 4 B is added into the MST
Step 6 C is added into the MST
Step 7 G is added into the MST
23
Notes about Prims algorithm
  • Proof by induction that this construction
    actually yields MST
  • Needs priority queue for locating closest fringe
    vertex
  • Efficiency
  • O(n2) for weight matrix representation of graph
    and array implementation of priority queue
  • O(m log n) for adjacency list representation of
    graph with n vertices and m edges and min-heap
    implementation of priority queue, how to get this

24
O(m log n) Prims Alg.
  • Hints
  • A mini-heap of size n, each vertex ordered by
    mini_dist of infinity except the initial vertex
  • parentn
  • n iterations of heap removal operation
  • For each removal, update the mini_dist and
    parent of the remaining vertices in the heap
  • m/n avg. of edges per vertex

25
Shortest paths Dijkstras algorithm
  • Single Source Shortest Paths Problem Given a
    weighted
  • connected graph G, find shortest paths from
    source vertex s
  • to each of the other vertices
  • Dijkstras algorithm Similar to Prims MST
    algorithm, with
  • a different way of computing numerical labels
    Among vertices
  • not already in the tree, it finds vertex u with
    the smallest sum
  • dv
    w(v,u)
  • where
  • v is a vertex for which shortest path has been
    already found on preceding iterations (such
    vertices form a tree)
  • dv is the length of the shortest path form
    source to v w(v,u) is the length (weight) of
    edge from v to u

26
Example
d
d
4
Tree vertices Remaining vertices
a(-,0) b(a,3) c(-,8) d(a,7) e(-,8)

4
b(a,3) c(b,34) d(b,32)
e(-,8)
b
c
3
6
5
2
a
d
e
7
4
4
d(b,5) c(b,7) e(d,54)
b
c
3
6
5
2
a
d
e
7
4
4
c(b,7) e(d,9)
b
c
3
6
2
5
a
d
e
7
4
e(d,9)
27
Notes on Dijkstras algorithm
  • Doesnt work for graphs with negative weights
  • Applicable to both undirected and directed graphs
  • Efficiency
  • O(V2) for graphs represented by weight matrix
    and array implementation of priority queue
  • O(ElogV) for graphs represented by adj. lists
    and min-heap implementation of priority queue
  • Dont mix up Dijkstras algorithm with Prims
    algorithm!

28
ReviewThe Knapsack Problem
  • The famous knapsack problem
  • A thief breaks into a museum. Fabulous
    paintings, sculptures, and jewels are everywhere.
    The thief has a good eye for the value of these
    objects, and knows that each will fetch hundreds
    or thousands of dollars on the clandestine art
    collectors market. But, the thief has only
    brought a single knapsack to the scene of the
    robbery, and can take away only what he can
    carry. What items should the thief take to
    maximize the haul?

29
Review The Knapsack Problem
  • More formally, the 0-1 knapsack problem
  • The thief must choose among n items, where the
    ith item worth vi dollars and weighs wi pounds
  • Carrying at most W pounds, maximize value
  • Note assume vi, wi, and W are all integers
  • 0-1 b/c each item must be taken or left in
    entirety
  • A variation, the fractional knapsack problem
  • Thief can take fractions of items
  • Think of items in 0-1 problem as gold ingots, in
    fractional problem as buckets of gold dust

30
Review The Knapsack Problem And Optimal
Substructure
  • Both variations exhibit optimal substructure
  • To show this for the 0-1 problem, consider the
    most valuable load weighing at most W pounds
  • If we remove item j from the load, what do we
    know about the remaining load?
  • A remainder must be the most valuable load
    weighing at most W - wj that thief could take
    from museum, excluding item j

31
Solving The Knapsack Problem
  • The optimal solution to the fractional knapsack
    problem can be found with a greedy algorithm
  • How?
  • The optimal solution to the 0-1 problem cannot be
    found with the same greedy strategy
  • Greedy strategy take in order of dollars/pound
  • Example 3 items weighing 10, 20, and 30 pounds,
    knapsack can hold 50 pounds
  • Suppose item 2 is worth 100. Assign values to
    the other items so that the greedy strategy will
    fail

32
(No Transcript)
33
The Knapsack Problem Greedy vs. DP
  • The fractional problem can be solved greedily
  • The 0-1 problem cannot be solved with a greedy
    approach
  • As you have seen, however, it can be solved with
    dynamic programming

34
Coding Problem
  • Coding assignment of bit strings to alphabet
    characters
  • Codewords bit strings assigned for characters of
    alphabet
  • Two types of codes
  • fixed-length encoding (e.g., ASCII)
  • variable-length encoding (e,g., Morse code)
  • Prefix-free codes no codeword is a prefix of
    another codeword
  • Problem If frequencies of the character
    occurrences are
  • known, what is the best binary
    prefix-free code?

35
Huffman codes
  • Any binary tree with edges labeled with 0s and
    1s yields a prefix-free code of characters
    assigned to its leaves
  • Optimal binary tree minimizing the expected
    (weighted average) length of a codeword can be
    constructed as follows
  • Huffmans algorithm
  • Initialize n one-node trees with alphabet
    characters and the tree weights with their
    frequencies.
  • Repeat the following step n-1 times join two
    binary trees with smallest weights into one (as
    left and right subtrees) and make its weight
    equal the sum of the weights of the two trees.
  • Mark edges leading to left and right subtrees
    with 0s and 1s, respectively.

36
Example
  • character A B C D _
  • frequency 0.35 0.1 0.2 0.2 0.15
  • codeword 11 100 00 01 101
  • average bits per character 2.25
  • for fixed-length encoding 3
  • compression ratio (3-2.25)/3100 25

37
(No Transcript)
38
(No Transcript)
39
(No Transcript)
Write a Comment
User Comments (0)
About PowerShow.com