COSC 3101A - Design and Analysis of Algorithms 10

1
COSC 3101A - Design and Analysis of Algorithms10

• BFS, DFS
• Topological Sort
• Strongly Connected Components

Many of these slides are taken from Monica
Nicolescu, Univ. of Nevada, Reno,
monica_at_cs.unr.edu
2
Searching in a Graph

• Graph searching systematically follow the edges
  of the graph so as to visit the vertices of the
  graph
• Two basic graph searching algorithms
• Breadth-first search
• Depth-first search
• The difference between them is in the order in
  which they explore the unvisited edges of the
  graph
• Graph algorithms are typically elaborations of
  the basic graph-searching algorithms

3
Breadth-First Search (BFS)

• Input
• A graph G (V, E) (directed or undirected)
• A source vertex s ? V
• Goal
• Explore the edges of G to discover every vertex
  reachable from s, taking the ones closest to s
  first
• Output
• dv distance (smallest of edges) from s to
  v, for all v ? V
• A breadth-first tree rooted at s that contains
  all reachable vertices

4
Breadth-First Search (cont.)

• Discover vertices in increasing order of distance
  from the source s search in breadth not depth
• Find all vertices at 1 edge from s, then all
  vertices at 2 edges from s, and so on

5
Breadth-First Search (cont.)

• Keeping track of progress
• Color each vertex in either white, gray or black
• Initially, all vertices are white
• When being discovered a vertex becomes gray
• After discovering all its adjacent vertices the
  node becomes black
• Use FIFO queue Q to maintain the set of gray
  vertices

source
6
Breadth-First Tree

• BFS constructs a breadth-first tree
• Initially contains the root (source vertex s)
• When vertex v is discovered while scanning the
  adjacency list of a vertex u ? vertex v and edge
  (u, v) are added to the tree
• u is the predecessor (parent) of v in the
  breadth-first tree
• A vertex is discovered only once ? it has at most
  one parent

source
7
BFS Additional Data Structures

• G (V, E) represented using adjacency lists
• coloru the color of the vertex for all u ? V
• ?u predecessor of u
• If u s (root) or node u has not yet been
  discovered ? ?u NIL
• du the distance from the source s to vertex u
• Use a FIFO queue Q to maintain the set of gray
  vertices

source
d1 ?1
d2 ?2
d1 ?1
d2 ?5
8
BFS(G, s)

1. for each u ? VG - s
2. do coloru ? WHITE
3. du ? ?
4. ?u NIL
5. colors ? GRAY
6. ds ? 0
7. ?s NIL
8. Q ? ?
9. Q ? ENQUEUE(Q, s)

Q s
9
BFS(G, s)

1. while Q ? ?
2. do u ? DEQUEUE(Q)
3. for each v ? Adju
4. do if colorv WHITE
5. then colorv ? GRAY
6. dv ? du 1
7. ?v u
8. ENQUEUE(Q, v)
9. coloru ? BLACK

Q s
Q w
Q w, r
10
Example
Q s
Q w, r
Q r, t, x
Q t, x, v
Q x, v, u
Q v, u, y
Q u, y
Q y
Q ?
11
Analysis of BFS

1. for each u ? V - s
2. do coloru ? WHITE
3. du ? ?
4. ?u NIL
5. colors ? GRAY
6. ds ? 0
7. ?s NIL
8. Q ? ?
9. Q ? ENQUEUE(Q, s)

O(V)
?(1)
12
Analysis of BFS

1. while Q ? ?
2. do u ? DEQUEUE(Q)
3. for each v ? Adju
4. do if colorv WHITE
5. then colorv GRAY
6. dv ? du 1
7. ?v u
8. ENQUEUE(Q, v)
9. coloru ? BLACK

?(1)
?(1)

• Total running time for BFS O(V E)

13
Shortest Paths Property

• BFS finds the shortest-path distance from the
  source vertex s ? V to each node in the graph
• Shortest-path distance ?(s, u)
• Minimum number of edges in any path from s to u

source
14
Depth-First Search

• Input
• G (V, E) (No source vertex given!)
• Goal
• Explore the edges of G to discover every vertex
  in V starting at the most current visited node
• Search may be repeated from multiple sources
• Output
• 2 timestamps on each vertex
• dv discovery time
• fv finishing time (done with examining vs
  adjacency list)
• Depth-first forest

15
Depth-First Search

• Search deeper in the graph whenever possible
• Edges are explored out of the most recently
  discovered vertex v that still has unexplored
  edges

• After all edges of v have been explored, the
  search backtracks from the parent of v
• The process continues until all vertices
  reachable from the original source have been
  discovered
• If undiscovered vertices remain, choose one of
  them as a new source and repeat the search from
  that vertex
• DFS creates a depth-first forest

16
DFS Additional Data Structures

• Global variable time-step
• Incremented when nodes are discovered/finished
• coloru similar to BFS
• White before discovery, gray while processing and
  black when finished processing
• ?u predecessor of u
• du, fu discovery and finish times

17
DFS(G)

• for each u ? VG
• do coloru ? WHITE
• ?u ? NIL
• time ? 0
• for each u ? VG
• do if coloru WHITE
• then DFS-VISIT(u)
• Every time DFS-VISIT(u) is called, u becomes the
  root of a new tree in the depth-first forest

18
DFS-VISIT(u)

1. coloru ? GRAY
2. time ? time1
3. du ? time
4. for each v ? Adju
5. do if colorv WHITE
6. then ?v ? u
7. DFS-VISIT(v)
8. coloru ? BLACK
9. time ? time 1
10. fu ? time

time 1
19
Example
20
Example (cont.)

• The results of DFS may depend on
• The order in which nodes are explored in
  procedure DFS
• The order in which the neighbors of a vertex are
  visited in DFS-VISIT

21
Edge Classification

• Tree edge (reaches a WHITE vertex)
• (u, v) is a tree edge if v was first discovered
  by exploring edge (u, v)
• Back edge (reaches a GRAY vertex)
• (u, v), connecting a vertex u to an ancestor v in
  a depth first tree
• Self loops (in directed graphs) are also back
  edges

22
Edge Classification

• Forward edge (reaches a BLACK vertex du lt
  dv)
• Non-tree edges (u, v) that connect a vertex u to
  a descendant v in a depth first tree
• Cross edge (reaches a BLACK vertex du gt
  dv)
• Can go between vertices in same depth-first tree
  (as long as there is no ancestor / descendant
  relation) or between different depth-first trees

23
Analysis of DFS(G)

• for each u ? VG
• do coloru ? WHITE
• ?u ? NIL
• time ? 0
• for each u ? VG
• do if coloru WHITE
• then DFS-VISIT(u)

24
Analysis of DFS-VISIT(u)

1. coloru ? GRAY
2. time ? time1
3. du ? time
4. for each v ? Adju
5. do if colorv WHITE
6. then ?v ? u
7. DFS-VISIT(v)
8. coloru ? BLACK
9. time ? time 1
10. fu ? time

DFS-VISIT is called exactly once for each vertex
Total Sv?V Adjv ?(V)
?(V E)
25
Properties of DFS

• u ?v ? DFS-VISIT(v) was called during a
  search of us adjacency list
• Vertex v is a descendant of vertex u in the depth
  first forest ? v is discovered during the time in
  which u is gray

26
Parenthesis Theorem
y
z
s
t

• In any DFS of a graph G, for all u, v, exactly
  one of the following holds
• du, fu and dv, fv are disjoint, and
  neither of u and v is a descendant of the other
• dv, fv is entirely within du, fu and
  v is a descendant of u
• du, fu is entirely within dv, fv and
  u is a descendant of v

3/6
2/9
1/10
11/16
4/5
7/8
12/13
14/15
x
u
v
w









s
t
v
z
u
w
y
x
1
2
3
4
5
6
7
8
9
10
13
11
12
14
15
16
(s
(z
(y
(x
x)
y)
(w
w)
z)
s)
u)
(t
(v
(u
u)
t)
Well-formed expression parenthesis are properly
nested
27
Other Properties of DFS

• Corollary
• Vertex v is a proper descendant of u
• ? du lt dv lt fv lt fu
• Theorem (White-path Theorem)
• In a depth-first forest of a graph G, vertex v
  is a descendant of u if and only if at time du,
  there is a path u ? v consisting of only white
  vertices.

28
Topological Sort

• Topological sort of a directed acyclic graph G
  (V, E) a linear order of vertices such that if
  there exists an edge (u, v), then u appears
  before v in the ordering.
• Directed acyclic graphs (DAGs)
• Used to represent precedence of events or
  processes that have a partial order
• a before b b before c
• b before c a before c

What about a and b?
a before c
Topological sort helps us establish a total order
29
Topological Sort

• TOPOLOGICAL-SORT(V, E)
• Call DFS(V, E) to compute finishing times fv
  for each vertex v
• When each vertex is finished, insert it onto the
  front of a linked list
• Return the linked list of vertices

undershorts
socks
11/
16
17/
18
15
12/
pants
shoes
13/
14
shirt
1/
8
6/
7
belt
watch
9/
10
tie
2/
5
jacket
3/
4
jacket
tie
belt
shirt
watch
shoes
pants
undershorts
socks
Running time ?(V E)
30
Topological Sort
undershorts
socks
11/
16
17/
18
Topological sort an ordering of vertices along a
horizontal line so that all directed edges go
from left to right.
15
12/
pants
shoes
13/
14
shirt
1/
8
6/
7
belt
watch
9/
10
tie
2/
5
jacket
3/
4
jacket
tie
belt
shirt
watch
shoes
pants
undershorts
socks
31
Lemma

• A directed graph is acyclic ? a DFS on G yields
  no back edges.
• Proof
• ? acyclic ? no back edge
• Assume back edge ? prove cycle
• Assume there is a back edge (u, v)
• v is an ancestor of u
• ? there is a path from v to u in G (v ? u)
• ? v ? u the back edge (u, v) yield a cycle

32
Lemma

• A directed graph is acyclic ? a DFS on G yields
  no back edges.
• Proof
• ? no back edge ? acyclic
• Assume cycle ? prove back edge
• Suppose G contains cycle c
• Let v be the first vertex discovered in c, and
  (u, v) be the preceding edge in c
• At time dv, vertices of c form a white path v ?
  u
• u is descendant of v in depth-first forest (by
  white-path theorem)
• ? (u, v) is a back edge

v
(u, v)
u
33
Strongly Connected Components

• Given directed graph G (V, E)
• A strongly connected component (SCC) of G is a
  maximal set of vertices C ? V such that for every
  pair of vertices u, v ? C, we have both u ?
  v and v ? u.

34
The Transpose of a Graph

• GT transpose of G
• GT is G with all edges reversed
• GT (V, ET), ET (u, v) (v, u) ? E
• If using adjacency lists we can create GT in
  ?(V E) time

35
Finding the SCC

• Observation G and GT have the same SCCs
• u and v are reachable from each other in G ? they
  are reachable from each other in GT
• Idea for computing the SCC of a DAG G (V, E)
• Make two depth first searches one on G and one
  on GT

36
STRONGLY-CONNECTED-COMPONENTS(G)

1. call DFS(G) to compute finishing times fu for
   each vertex u
2. compute GT
3. call DFS(GT), but in the main loop of DFS,
   consider vertices in order of decreasing fu (as
   computed in first DFS)
4. output the vertices in each tree of the
   depth-first forest formed in second DFS as a
   separate SCC

37
Example
DFS on the initial graph G
1/
8/
11/
13/
9
10
14
16
f 4
h 6
g 7
d 9
c 10
a 14
e 15
b 16
2/
3/
4
6
5/
7
12/
15

• DFS on GT
• start at b visit a, e
• start at c visit d
• start at g visit f
• start at h

Strongly connected components C1 a, b, e, C2
c, d, C3 f, g, C4 h
38
Component Graph

• The component graph GSCC (VSCC, ESCC)
• VSCC v1, v2, , vk, where vi corresponds to
  each strongly connected component Ci
• There is an edge (vi, vj) ? ESCC if G contains a
  directed edge (x, y) for some x ? Ci and y ? Cj
• The component graph is a DAG

39
Lemma 1

• Let C and C be distinct SCCs in G
• Let u, v ? C, and u, v ? C
• Suppose there is a path u ? u in G
• Then there cannot also be a path v ? v in G.

• Proof
• Suppose there is a path v ? v
• There exists u ? u ? v
• There exists v ? v ? u
• u and v are reachable from each other, so they
  are not in separate SCCs contradiction!

C
C
u
u
v
v
40
Notations

• Extend notation for d (starting time) and f
  (finishing time) to sets of vertices U ? V
• d(U) minu?U du (earliest discovery time)
• f(U) maxu?U fu (latest finishing time)

C1
C2
a
b
c
d
1/
8/
11/
13/
9
10
14
16
2/
3/
4
6
5/
7
12/
15
e
f
g
h
C4
C3
41
Lemma 2

• Let C and C be distinct SCCs in a directed graph
  G (V, E). If there is an edge (u, v) ? E, where
  u ? C and v ? C then f(C) gt f(C).
• Consider C1 and C2, connected by edge (b, c)

C1
C2
a
b
c
d
1/
8/
11/
13/
9
10
14
16
2/
3/
4
6
5/
7
12/
15
e
f
g
h
C4
C3
42
Corollary

• Let C and C be distinct SCCs in a directed graph
  G (V, E). If there is an edge (u, v) ? ET,
  where u ? C and v ? C then f(C) lt f(C).
• Consider C2 and C1, connected by edge (c, b)

C1 C
C2 C

• Since (c, b) ? ET ? (b, c) ? E
• From previous lemma
• f(C1) gt f(C2)
• f(C) gt f(C)

C4
C3
43
Discussion

• f(C) lt f(C)
• Each edge in GT that goes between different
  components goes from a component with an earlier
  finish time (in the DFS) to one with a later
  finish time

C1 C
C2 C
C4
C3
44
Why does SCC Work?

• When we do the second DFS, on GT, we start with a
  component C such that f(C) is maximum (b, in our
  case)
• We start from b and visit all vertices in C1
• From corollary f(C) gt f(C) for all C ? C ?
  there are no edges from C to any other SCCs in GT
• ? DFS will visit only vertices in C1
• ? The depth-first tree rooted at b contains
  exactly the vertices of C1

C1
C2
C3
C4
45
Why does SCC Work? (cont.)

• The next root chosen in the second DFS is in SCC
  C2 such that f(C) is maximum over all SCCs other
  than C1
• DFS visits all vertices in C2
• the only edges out of C2 go to C1, which weve
  already visited
• ? The only tree edges will be to vertices in C2
• Each time we choose a new root it can reach only
• vertices in its own component
• vertices in components already visited

C1
C2
C3
C4
46
Readings

• Chapter 22
• Appendix B