Title: CSE 326: Data Structures Part 8 Graphs
1CSE 326 Data StructuresPart 8Graphs
- Henry Kautz
- Autumn Quarter 2002
2Outline
- Graphs (TO DO READ WEISS CH 9)
- Graph Data Structures
- Graph Properties
- Topological Sort
- Graph Traversals
- Depth First Search
- Breadth First Search
- Iterative Deepening Depth First
- Shortest Path Problem
- Dijkstras Algorithm
3Graph ADT
- Graphs are a formalism for representing
relationships between objects - a graph G is represented as G (V, E)
- V is a set of vertices
- E is a set of edges
- operations include
- iterating over vertices
- iterating over edges
- iterating over vertices adjacent to a specific
vertex - asking whether an edge exists connected two
vertices
V Han, Leia, Luke E (Luke, Leia),
(Han, Leia), (Leia, Han)
4What Graph is THIS?
5ReferralWeb(co-authorship in scientific papers)
6Biological Function Semantic Network
7Graph Representation 1 Adjacency Matrix
- A V x V array in which an element (u, v) is
true if and only if there is an edge from u to v
Han
Luke
Leia
Han
Luke
Runtime iterate over vertices iterate ever
edges iterate edges adj. to vertex edge exists?
Leia
Space requirements
8Graph Representation 2 Adjacency List
- A V-ary list (array) in which each entry stores
a list (linked list) of all adjacent vertices
Han
Luke
Runtime iterate over vertices iterate ever
edges iterate edges adj. to vertex edge exists?
Leia
space requirements
9Directed vs. Undirected Graphs
- In directed graphs, edges have a specific
direction - In undirected graphs, they dont (edges are
two-way) - Vertices u and v are adjacent if (u, v) ? E
Han
Luke
Leia
Han
Luke
Leia
10Graph Density
- A sparse graph has O(V) edges
- A dense graph has ?(V2) edges
- Anything in between is either sparsish or densy
depending on the context.
11Weighted Graphs
Each edge has an associated weight or cost.
20
Clinton
Mukilteo
30
Kingston
Edmonds
35
Bainbridge
Seattle
60
Bremerton
There may be more information in the graph as
well.
12Paths and Cycles
- A path is a list of vertices v1, v2, , vn such
that (vi, vi1) ? E for all 0 ? i lt n. - A cycle is a path that begins and ends at the
same node.
Chicago
Seattle
Salt Lake City
San Francisco
Dallas
p Seattle, Salt Lake City, Chicago, Dallas,
San Francisco, Seattle
13Path Length and Cost
- Path length the number of edges in the path
- Path cost the sum of the costs of each edge
Chicago
3.5
Seattle
2
2
Salt Lake City
2
2.5
2.5
2.5
3
San Francisco
Dallas
length(p) 5
cost(p) 11.5
14Connectivity
- Undirected graphs are connected if there is a
path between any two vertices - Directed graphs are strongly connected if there
is a path from any one vertex to any other - Directed graphs are weakly connected if there is
a path between any two vertices, ignoring
direction - A complete graph has an edge between every pair
of vertices
15Trees as Graphs
- Every tree is a graph with some restrictions
- the tree is directed
- there are no cycles (directed or undirected)
- there is a directed path from the root to every
node
A
B
C
D
E
F
H
G
BAD!
J
I
16Directed Acyclic Graphs (DAGs)
- DAGs are directed graphs with no cycles.
main()
mult()
if program call graph is a DAG, then all
procedure calls can be in-lined
add()
read()
access()
Trees ? DAGs ? Graphs
17Application of DAGs Representing Partial Orders
reserve flight
check in airport
call taxi
pack bags
take flight
locate gate
taxi to airport
18Topological Sort
- Given a graph, G (V, E), output all the
vertices in V such that no vertex is output
before any other vertex with an edge to it.
reserve flight
check in airport
call taxi
take flight
taxi to airport
locate gate
pack bags
19Topo-Sort Take One
- Label each vertexs in-degree ( of inbound
edges) - While there are vertices remaining
- Pick a vertex with in-degree of zero and output
it - Reduce the in-degree of all vertices adjacent to
it - Remove it from the list of vertices
runtime
20Topo-Sort Take Two
- Label each vertexs in-degree
- Initialize a queue (or stack) to contain all
in-degree zero vertices - While there are vertices remaining in the queue
- Remove a vertex v with in-degree of zero and
output it - Reduce the in-degree of all vertices adjacent to
v - Put any of these with new in-degree zero on the
queue
runtime
21Recall Tree Traversals
a
e
d
b
c
i
h
j
f
g
k
l
a b f g k c d h i l j e
22Depth-First Search
- Pre/Post/In order traversals are examples of
depth-first search - Nodes are visited deeply on the left-most
branches before any nodes are visited on the
right-most branches - Visiting the right branches deeply before the
left would still be depth-first! Crucial idea is
go deep first! - Difference in pre/post/in-order is how some
computation (e.g. printing) is done at current
node relative to the recursive calls - In DFS the nodes being worked on are kept on a
stack
23Iterative Version DFSPre-order Traversal
- Push root on a Stack
- Repeat until Stack is empty
- Pop a node
- Process it
- Push its children on the Stack
24Level-Order Tree Traversal
- Consider task of traversing tree level by level
from top to bottom (alphabetic order) - Is this also DFS?
25Breadth-First Search
- No! Level-order traversal is an example of
Breadth-First Search - BFS characteristics
- Nodes being worked on maintained in a FIFO Queue,
not a stack - Iterative style procedures often easier to design
than recursive procedures - Put root in a Queue
- Repeat until Queue is empty
- Dequeue a node
- Process it
- Add its children to queue
26QUEUE
- a
- b c d e
- c d e f g
- d e f g
- e f g h i j
- f g h i j
- g h i j
- h i j k
- i j k
- j k l
- k l
- l
27Graph Traversals
- Depth first search and breadth first search also
work for arbitrary (directed or undirected)
graphs - Must mark visited vertices so you do not go into
an infinite loop! - Either can be used to determine connectivity
- Is there a path between two given vertices?
- Is the graph (weakly) connected?
- Important difference Breadth-first search
always finds a shortest path from the start
vertex to any other (for unweighted graphs) - Depth first search may not!
28Demos on Web PageDFSBFS
29Is BFS the Hands Down Winner?
- Depth-first search
- Simple to implement (implicit or explict stack)
- Does not always find shortest paths
- Must be careful to mark visited vertices, or
you could go into an infinite loop if there is a
cycle - Breadth-first search
- Simple to implement (queue)
- Always finds shortest paths
- Marking visited nodes can improve efficiency, but
even without doing so search is guaranteed to
terminate
30Space Requirements
- Consider space required by the stack or queue
- Suppose
- G is known to be at distance d from S
- Each vertex n has k out-edges
- There are no (undirected or directed) cycles
- BFS queue will grow to size kd
- Will simultaneously contain all nodes that are at
distance d (once last vertex at distance d-1 is
expanded) - For k10, d15, size is 1,000,000,000,000,000
31DFS Space Requirements
- Consider DFS, where we limit the depth of the
search to d - Force a backtrack at d1
- When visiting a node n at depth d, stack will
contain - (at most) k-1 siblings of n
- parent of n
- siblings of parent of n
- grandparent of n
- siblings of grandparent of n
- DFS queue grows at most to size dk
- For k10, d15, size is 150
- Compare with BFS 1,000,000,000,000,000
32Conclusion
- For very large graphs DFS is hugely more memory
efficient, if we know the distance to the goal
vertex! - But suppose we dont know d. What is the
(obvious) strategy?
33Iterative Deepening DFS
- IterativeDeepeningDFS(vertex s, g)
- for (i1truei)
- if DFS(i, s, g) return
-
- // Also need to keep track of path found
- bool DFS(int limit, vertex s, g)
- if (sg) return true
- if (limit-- lt 0) return false
- for (n in children(s))
- if (DFS(limit, n, g)) return true
- return false
34Analysis of Iterative Deepening
- Even without marking nodes as visited,
iterative-deepening DFS never goes into an
infinite loop - For very large graphs, memory cost of keeping
track of visited vertices may make marking
prohibitive - Work performed with limit lt actual distance to G
is wasted but the wasted work is usually small
compared to amount of work done during the last
iteration
35Asymptotic Analysis
- There are pathological graphs for which
iterative deepening is bad
nd
G
S
36A Better Case
- Suppose each vertex n has k out-edges, no cycles
- Bounded DFS to level i reaches ki vertices
- Iterative Deepening DFS(d)
ignore low order terms!
37(More) Conclusions
- To find a shortest path between two nodes in a
unweighted graph, use either BFS or Iterated DFS - If the graph is large, Iterated DFS typically
uses much less memory - Later well learn about heuristic search
algorithms, which use additional knowledge about
the problem domain to reduce the number of
vertices visited
38Single Source, Shortest Path for Weighted Graphs
- Given a graph G (V, E) with edge costs c(e),
and a vertex s ? V, find the shortest (lowest
cost) path from s to every vertex in V - Graph may be directed or undirected
- Graph may or may not contain cycles
- Weights may be all positive or not
- What is the problem if graph contains cycles
whose total cost is negative?
39The Trouble with Negative Weighted Cycles
2
A
B
10
-5
1
E
2
C
D
40Edsger Wybe Dijkstra (1930-2002)
- Invented concepts of structured programming,
synchronization, weakest precondition, and
"semaphores" for controlling computer processes.
The Oxford English Dictionary cites his use of
the words "vector" and "stack" in a computing
context. - Believed programming should be taught without
computers - 1972 Turing Award
- In their capacity as a tool, computers will be
but a ripple on the surface of our culture. In
their capacity as intellectual challenge, they
are without precedent in the cultural history of
mankind.
41Dijkstras Algorithm for Single Source Shortest
Path
- Classic algorithm for solving shortest path in
weighted graphs (with only positive edge weights) - Similar to breadth-first search, but uses a
priority queue instead of a FIFO queue - Always select (expand) the vertex that has a
lowest-cost path to the start vertex - a kind of greedy algorithm
- Correctly handles the case where the lowest-cost
(shortest) path to a vertex is not the one with
fewest edges
42Pseudocode for Dijkstra
- Initialize the cost of each vertex to ?
- costs 0
- heap.insert(s)
- While (! heap.empty())
- n heap.deleteMin()
- For (each vertex a which is adjacent to n along
edge e) - if (costn edge_coste lt costa) then
- cost a costn edge_coste
- previous_on_path_toa n
- if (a is in the heap) then heap.decreaseKey(a)
- else heap.insert(a)
43Important Features
- Once a vertex is removed from the head, the cost
of the shortest path to that node is known - While a vertex is still in the heap, another
shorter path to it might still be found - The shortest path itself from s to any node a can
be found by following the pointers stored in
previous_on_path_toa
44Dijkstras Algorithm in Action
45DemoDijkstras
46Data Structures for Dijkstras Algorithm
V times
Select the unknown node with the lowest cost
findMin/deleteMin
O(log V)
E times
as cost min(as old cost, )
decreaseKey
O(log V)
runtime O(E log V)
47CSE 326 Data StructuresLecture 8.BHeuristic
Graph Search
- Henry Kautz
- Winter Quarter 2002
48Homework Hint - Problem 4
- You can turn in a final version of your answer to
problem 4 without penalty on Wednesday.
49Outline
- Best First Search
- A Search
- Example Plan Synthesis
- This material is NOT in Weiss, but is important
for both the programming project and the final
exam!
50Huge Graphs
- Consider some really huge graphs
- All cities and towns in the World Atlas
- All stars in the Galaxy
- All ways 10 blocks can be stacked
- Huh???
51Implicitly Generated Graphs
- A huge graph may be implicitly specified by rules
for generating it on-the-fly - Blocks world
- vertex relative positions of all blocks
- edge robot arm stacks one block
stack(blue,table)
stack(green,blue)
stack(blue,red)
stack(green,red)
stack(green,blue)
52Blocks World
- Source initial state of the blocks
- Goal desired state of the blocks
- Path source to goal sequence of actions
(program) for robot arm! - n blocks ? nn vertices
- 10 blocks ? 10 billion vertices!
53Problem Branching Factor
- Cannot search such huge graphs exhaustively.
Suppose we know that goal is only d steps away. - Dijkstras algorithm is basically breadth-first
search (modified to handle arc weights) - Breadth-first search (or for weighted graphs,
Dijkstras algorithm) If out-degree of each
node is 10, potentially visits 10d vertices - 10 step plan 10 billion vertices visited!
54An Easier Case
- Suppose you live in Manhattan what do you do?
S
52nd St
G
51st St
50th St
10th Ave
9th Ave
8th Ave
7th Ave
6th Ave
5th Ave
4th Ave
3rd Ave
2nd Ave
55Best-First Search
- The Manhattan distance (? x ? y) is an estimate
of the distance to the goal - a heuristic value
- Best-First Search
- Order nodes in priority to minimize estimated
distance to the goal h(n) - Compare BFS / Dijkstra
- Order nodes in priority to minimize distance from
the start
56Best First in Action
- Suppose you live in Manhattan what do you do?
S
52nd St
G
51st St
50th St
10th Ave
9th Ave
8th Ave
7th Ave
6th Ave
5th Ave
4th Ave
3rd Ave
2nd Ave
57Problem 1 Led Astray
- Eventually will expand vertex to get back on the
right track
S
G
52nd St
51st St
50th St
10th Ave
9th Ave
8th Ave
7th Ave
6th Ave
5th Ave
4th Ave
3rd Ave
2nd Ave
58Problem 2 Optimality
- With Best-First Search, are you guaranteed a
shortest path is found when - goal is first seen?
- when goal is removed from priority queue (as with
Dijkstra?)
59Sub-Optimal Solution
- No! Goal is by definition at distance 0 will be
removed from priority queue immediately, even if
a shorter path exists!
(5 blocks)
S
52nd St
h2
h5
G
h4
51st St
h1
9th Ave
8th Ave
7th Ave
6th Ave
5th Ave
4th Ave
60Synergy?
- Dijkstra / Breadth First guaranteed to find
optimal solution - Best First often visits far fewer vertices, but
may not provide optimal solution - Can we get the best of both?
61A (A star)
- Order vertices in priority queue to minimize
- (distance from start) (estimated distance to
goal) - f(n) g(n) h(n)
- f(n) priority of a node
- g(n) true distance from start
- h(n) heuristic distance to goal
62Optimality
- Suppose the estimated distance (h) is always
less than or equal to the true distance to the
goal - heuristic is a lower bound on true distance
- Then when the goal is removed from the priority
queue, we are guaranteed to have found a shortest
path!
63Problem 2 Revisited
vertex g(n) h(n) f(n)
52nd 9th 0 5 5
S
(5 blocks)
52nd St
G
51st St
50th St
9th Ave
8th Ave
7th Ave
6th Ave
5th Ave
4th Ave
64Problem 2 Revisited
vertex g(n) h(n) f(n)
52nd 4th 5 2 7
51st 9th 1 4 5
S
(5 blocks)
52nd St
G
51st St
50th St
9th Ave
8th Ave
7th Ave
6th Ave
5th Ave
4th Ave
65Problem 2 Revisited
vertex g(n) h(n) f(n)
52nd 4th 5 2 7
51st 8th 2 3 5
50th 9th 2 5 7
S
(5 blocks)
52nd St
G
51st St
50th St
9th Ave
8th Ave
7th Ave
6th Ave
5th Ave
4th Ave
66Problem 2 Revisited
vertex g(n) h(n) f(n)
52nd 4th 5 2 7
51st 7th 3 2 5
50th 9th 2 5 7
50th 8th 3 4 7
S
(5 blocks)
52nd St
G
51st St
50th St
9th Ave
8th Ave
7th Ave
6th Ave
5th Ave
4th Ave
67Problem 2 Revisited
vertex g(n) h(n) f(n)
52nd 4th 5 2 7
51st 6th 4 1 5
50th 9th 2 5 7
50th 8th 3 4 7
50th 7th 4 3 7
S
(5 blocks)
52nd St
G
51st St
50th St
9th Ave
8th Ave
7th Ave
6th Ave
5th Ave
4th Ave
68Problem 2 Revisited
vertex g(n) h(n) f(n)
52nd 4th 5 2 7
51st 5th 5 0 5
50th 9th 2 5 7
50th 8th 3 4 7
50th 7th 4 3 7
S
(5 blocks)
52nd St
G
51st St
50th St
9th Ave
8th Ave
7th Ave
6th Ave
5th Ave
4th Ave
69Problem 2 Revisited
vertex g(n) h(n) f(n)
52nd 4th 5 2 7
50th 9th 2 5 7
50th 8th 3 4 7
50th 7th 4 3 7
S
(5 blocks)
52nd St
G
51st St
50th St
9th Ave
8th Ave
7th Ave
6th Ave
5th Ave
4th Ave
DONE!
70What Would Dijkstra Have Done?
S
(5 blocks)
52nd St
G
51st St
50th St
49th St
48th St
47th St
9th Ave
8th Ave
7th Ave
6th Ave
5th Ave
4th Ave
71Proof of A Optimality
- A terminates when G is popped from the heap.
- Suppose G is popped but the path found isnt
optimal - priority(G) gt optimal path length c
- Let P be an optimal path from S to G, and let N
be the last vertex on that path that has been
visited but not yet popped. - There must be such an N, otherwise the optimal
path would have been found. - priority(N) g(N) h(N) ? c
- So N should have popped before G can pop.
Contradiction.
non-optimal path to G
S
G
undiscovered portion of shortest path
portion of optimal path found so far
N
72What About Those Blocks?
- Distance to goal is not always physical
distance - Blocks world
- distance number of stacks to perform
- heuristic lower bound number of blocks out of
place
out of place 2, true distance to goal 3
733-Blocks State Space Graph
ABCh2
CABh3
BACh2
ABCh1
CBAh3
A CBh2
BCAh1
BC Ah3
CB Ah3
C A Bh3
AC Bh3
BA Ch2
AB Ch0
start
goal
743-Blocks Best First Solution
ABCh2
CABh3
BACh2
ABCh1
CBAh3
A CBh2
BCAh1
BC Ah3
CB Ah3
C A Bh3
AC Bh3
BA Ch2
AB Ch0
start
goal
753-Blocks BFS Solution
ABCh2
expanded, but not in solution
CABh3
BACh2
ABCh1
CBAh3
A CBh2
BCAh1
BC Ah3
CB Ah3
C A Bh3
AC Bh3
BA Ch2
AB Ch0
start
goal
763-Blocks A Solution
ABCh2
expanded, but not in solution
CABh3
BACh2
ABCh1
CBAh3
A CBh2
BCAh1
BC Ah3
CB Ah3
C A Bh3
AC Bh3
BA Ch2
AB Ch0
start
goal
77Other Real-World Applications
- Routing finding computer networks, airline
route planning - VLSI layout cell layout and channel routing
- Production planning just in time optimization
- Protein sequence alignment
- Many other NP-Hard problems
- A class of problems for which no exact polynomial
time algorithms exist so heuristic search is
the best we can hope for
78Coming Up
- Other graph problems
- Connected components
- Spanning tree
79CSE 326 Data StructuresPart 8.CSpanning Trees
and More
- Henry Kautz
- Autumn Quarter 2002
80Today
- Incremental hashing
- MazeRunner project
- Longest Path?
- Finding Connected Components
- Application to machine vision
- Finding Minimum Spanning Trees
- Yet another use for union/find
81Incremental Hashing
82Maze Runner
20 15 --------------------
---- - - -
-- ------
-----------
- X
- -- --- - - ------
- - - -
--
-
-
---- -
-
-- --
------------
-------
- ----
---
- - - -
----
- --- -- - -
-
--------------------
- DFS, iterated DFS, BFS, best-first, A
- Crufty old C code from fresh clean Java code
- Win fame and glory by writing a nice real-time
maze visualizer
83Java Note
Java lacks enumerated constants enum DOG,
CAT, MOUSE animal animal a DOG Static
constants not type-safe static final int DOG
1 static final int CAT 2 static final int
BLUE 1 int favoriteColor DOG
84Amazing Java Trick
public final class Animal private Animal()
public static final Animal DOG new Animal()
public static final Animal CAT new
Animal() public final class Color private
Color() public static final Animal BLUE new
Color() Animal x DOG Animal x BLUE //
Gives compile-time error!
85Longest Path Problem
- Given a graph G(V,E) and vertices s, t
- Find a longest simple path (no repeating
vertices) from s to t. - Does reverse Dijkstra work?
86Dijkstra
- Initialize the cost of each vertex to ?
- costs 0
- heap.insert(s)
- While (! heap.empty())
- n heap.deleteMin()
- For (each vertex a which is adjacent to n along
edge e) - if (costn edge_coste lt costa) then
- cost a costn edge_coste
- previous_on_path_toa n
- if (a is in the heap) then heap.decreaseKey(a)
- else heap.insert(a)
87Reverse Dijkstra
- Initialize the cost of each vertex to ?
- costs 0
- heap.insert(s)
- While (! heap.empty())
- n heap.deleteMax()
- For (each vertex a which is adjacent to n along
edge e) - if (costn edge_coste gt costa) then
- cost a costn edge_coste
- previous_on_path_toa n
- if (a is in the heap) then heap.increaseKey(a)
- else heap.insert(a)
88Does it Work?
a
6
t
5
3
b
1
s
89Problem
- No clear stopping condition!
- How many times could a vertex be inserted in the
priority queue? - Exponential!
- Not a good algorithm!
- Is the better one?
90Counting Connected Components
- Initialize the cost of each vertex to ?
- Num_cc 0
- While there are vertices of cost ?
- Pick an arbitrary such vertex S, set its cost to
0 - Find paths from S
- Num_cc
91Using DFS
- Set each vertex to unvisited
- Num_cc 0
- While there are unvisited vertices
- Pick an arbitrary such vertex S
- Perform DFS from S, marking vertices as visited
- Num_cc
Complexity O(VE)
92Using Union / Find
- Put each node in its own equivalence class
- Num_cc 0
- For each edge E ltx,ygt
- Union(x,y)
- Return number of equivalence classes
Complexity
93Using Union / Find
- Put each node in its own equivalence class
- Num_cc 0
- For each edge E ltx,ygt
- Union(x,y)
- Return number of equivalence classes
Complexity O(VE ack(E,V))
94Machine Vision Blob Finding
95Machine Vision Blob Finding
1
2
3
5
4
96Blob Finding
- Matrix can be considered an efficient
representation of a graph with a very regular
structure - Cell vertex
- Adjacent cells of same color edge between
vertices - Blob finding finding connected components
97Tradeoffs
- Both DFS and Union/Find approaches are
(essentially) O(EV) O(E) for binary
images - For each component, DFS (recursive labeling)
can move all over the image entire image must
be in main memory - Better in practice row-by-row processing
- localizes accesses to memory
- typically 1-2 orders of magnitude faster!
98High-Level Blob-Labeling
- Scan through image left/right and top/bottom
- If a cell is same color as (connected to) cell to
right or below, then union them - Give the same blob number to cells in each
equivalence class
99Blob-Labeling Algorithm
- Put each cell ltx,ygt in its own equivalence class
- For each cell ltx,ygt
- if colorx,y colorx1,y then
- Union( ltx,ygt, ltx1,ygt )
- if colorx,y colorx,y1 then
- Union( ltx,ygt, ltx,y1gt )
- label 0
- For each root ltx,ygt
- blobnumx,y label
- For each cell ltx,ygt
- blobnumx,y blobnum( Find(ltx,ygt) )
100Spanning Tree
- Spanning tree a subset of the edges from a
connected graph that - touches all vertices in the graph (spans the
graph) - forms a tree (is connected and contains no
cycles) - Minimum spanning tree the spanning tree with the
least total edge cost.
4
7
9
2
1
5
101Applications of Minimal Spanning Trees
- Communication networks
- VLSI design
- Transportation systems
102Kruskals Algorithm for Minimum Spanning Trees
- A greedy algorithm
- Initialize all vertices to unconnected
- While there are still unmarked edges
- Pick a lowest cost edge e (u, v) and mark it
- If u and v are not already connected, add e to
the minimum spanning tree and connect u and v
Sound familiar? (Think maze generation.)
103Kruskals Algorithm in Action (1/5)
2
2
3
B
A
F
H
2
1
1
10
4
9
G
4
C
8
2
D
E
7
104Kruskals Algorithm in Action (2/5)
2
2
3
B
A
F
H
2
1
1
10
4
9
G
4
C
8
2
D
E
7
105Kruskals Algorithm in Action (3/5)
2
2
3
B
A
F
H
2
1
1
10
4
9
G
4
C
8
2
D
E
7
106Kruskals Algorithm in Action (4/5)
2
2
3
B
A
F
H
2
1
1
10
4
9
G
4
C
8
2
D
E
7
107Kruskals Algorithm Completed (5/5)
2
2
3
B
A
F
H
2
1
1
10
4
9
G
4
C
8
2
D
E
7
108Why Greediness Works
- Proof by contradiction that Kruskals finds a
minimum spanning tree - Assume another spanning tree has lower cost than
Kruskals. - Pick an edge e1 (u, v) in that tree thats not
in Kruskals. - Consider the point in Kruskals algorithm where
us set and vs set were about to be connected.
Kruskal selected some edge to connect them call
it e2 . - But, e2 must have at most the same cost as e1
(otherwise Kruskal would have selected it
instead). - So, swap e2 for e1 (at worst keeping the cost the
same) - Repeat until the tree is identical to Kruskals,
where the cost is the same or lower than the
original cost contradiction!
109Data Structures for Kruskals Algorithm
Once
E times
Initialize heap of edges
Pick the lowest cost edge
buildHeap
findMin/deleteMin
E times
If u and v are not already connected connect u
and v.
union
runtime
E E log E E ack(E,V)
110Data Structures for Kruskals Algorithm
Once
E times
Initialize heap of edges
Pick the lowest cost edge
buildHeap
findMin/deleteMin
E times
If u and v are not already connected connect u
and v.
union
runtime
E E log E E ack(E,V)
O(ElogE)
111Prims Algorithm
- Can also find Minimum Spanning Trees using a
variation of Dijkstras algorithm - Pick a initial node
- Until graph is connected
- Choose edge (u,v) which is of minimum cost among
edges where u is in tree but v is not - Add (u,v) to the tree
- Same greedy proof, same asymptotic complexity
112Coming Up
- Application Sentence Disambiguation
- All-pairs Shortest Paths
- NP-Complete Problems
- Advanced topics
- Quad trees
- Randomized algorithms
113Sentence Disambiguation
- A person types a message on their cell phone
keypad. Each button can stand for three
different letter (e.g. 1 is a, b, or c), but
the person does not explicitly indicate which
letter is meant. (Words are separated by blanks
the 0 key.) - Problem How can the system determine what
sentence was typed? - My Nokia cell phone does this!
- How can this problem be cast as a shortest-path
problem?
114(No Transcript)
115Sentence Disambiguation as Shortest Path
- Idea
- Possible words are vertices
- Directed edge between adjacent possible words
- Weight on edge from W1 to W2 is probability that
W2 appears adjacent to W1 - Probabilities over what?! Some large archive
(corpus) of text - Word bi-gram model
- Find the most probable path through the graph
116W11
W12
W13
W21
W23
W22
W11
W31
W33
W41
W43
117Technical Concerns
- Isnt most probable actually longest (most
heavily weighted) path?! - Shouldnt we be multiplying probabilities, not
adding them?!
118Logs to the Rescue
- Make weight on edge fromW1 to W2 be
- - log P(W2 W1)
- Logs of probabilities are always negative
numbers, so take negative logs - The lower the probability, the larger the
negative log! So this is shortest path - Adding logs is the same as multiplying the
underlying quantities
119To Think About
- This really works in practice 99 accuracy!
- Cell phone memory is limited how can we use as
little storage as possible? - How can the system customize itself to a user?
120Question
- Which graph algorithm is asymptotically better
- ?(VElogV)
- ?(V3)
121All Pairs Shortest Path
- Suppose you want to compute the length of the
shortest paths between all pairs of vertices in a
graph - Run Dijkstras algorithm (with priority queue)
repeatedly, starting with each node in the graph - Complexity in terms of V when graph is dense
122Dynamic Programming Approach
123Floyd-Warshall Algorithm
- // C adjacency matrix representation of graph
- // Cij weighted edge i-gtj or ? if
none - // D computed distances
- FW(int n, int C , int D )
- for (i 0 i lt N i)
- for (j 0 j lt N j)
- Dij Cij
- Dii 0.0
-
- for (k 0 k lt N k)
- for (i 0 i lt N i)
- for (j 0 j lt N j)
- if (Dik Dkj lt Dij)
- Dij Dik Dkj
Run time
How could we compute the paths?