Probability Proofs Graphs Master Theorem

1
Probability ProofsGraphsMaster Theorem
2
Probability Terms

• Probability space a set S with a function p that
  takes elements of S to nonnegative real numbers.
  p(s) is called the probability of s.
  Probabilities must sum to 1.0.
• Example S H, T, and p(H) p(T) 0.5. This
  is a model of a fair coin.
• Event Any subset V of a probability space.
• Probability of an event the sum of the
  probabilities of the elements of V.

3
Probability

• Lets return to the IP address example. Lets
  take 2-number addresses (x1, x2), and assume that
  the numbers run from 0 to 4
• Our hash function is
• ha(x) a1x1 a2 x2 mod 5
• We choose a1 and a2 at random, 0 to 4

4
Set of hash functions

• H ha 0
• Elements of H correspond to (0,0), (0, 1), (0,2),
  (0, 3), (0, 4), , (4, 4)
• There are 25 of these
• All are assigned equal probability
• This makes H a probability space a set together
  with an assignment of numbers called
  probabilities.
• Sum of probabilities is 1.0

5
Example Events

• An event is a subset V of a probability space.
• The probability of V is defined to be the sum of
  the probabilities of the elements of V.
• Example in H, the event a1 0 consists of five
  elements (0,0), (0,1), (0, 2), (0, 3), (0, 4),
  each of probability 1/25
• Pra1 0 5 . (1/25) 1/5 20.

6
Another event

• Consider all hash functions in H. We draw one at
  random and use it to hash (x1, x2) (3, 2).
• The event V consists of all hash functions for
  which this resulting hash value is 0
• i.e., V ha 3a1 2a2 0
• This set, too, has five elements, so PrV 20.

7
One more event

• V is the set of hash functions with ha(1,0) 0
  or ha(0, 1) 0
• This means that either a1 0 or a2 0
• There are 9 such (a1, a2) pairs
• So PrV 9/25.

8
Event we cared about

• We had a pair of distinct IP addresses x and y
• They were 4 bytes each well consider two-term
  IP addresses, with each term in 0, 4
• Assume x2 different from y2
• We wanted to know
• Pra1 x1 a2 x2 a1 y1 a2 y2 mod 5
• This (informally) defines a set of (a1, a2) pairs

9
Pra1 x1 a2 x2 a1 y1 a2 y2 mod 5

• Rewrite as
• a1 (x1 y1) a2 (y2 x2) mod 5
• For each (a1, a2) pair satisfying this, compute
  b a1 (x1 y1), and q (y2 x2)
• Equation above is
• b a2 q mod 5
• where q is nonzero
• Theres exactly one number a2 making this true
• So the event has exactly five members, and
  Prevent 20.

10
GRAPHS!

• Occur everywhere
• Internet
• Facebook friends graph
• Representations of explorable space for robot
• Representations of geometric relationships in
  graphics
• graphical models in AI (key to speech
  recognition, some hedge funds, )

11
Terminology

• Collection of nodes (or vertices) and edges.
• Vertex set denoted V edge set denoted E.
• Sometimes V(G) or E(G) if needed for clarity
• V is number of nodes E is number of edges
• V A,B,C E A,C, B, C

12
Kinds of graphs

• Multigraph multiple edges from p to q allowed
• Simple no edge from n to n (i.e., no loops) and
  no multi-edges
• Directed edges have arrows on them
• Weighted edges have weights or lengths on
  them
• Usually called lengths if positive, weights if
  some are negative
• For now simple, undirected, unweighted graphs

13
Data Representations

• Adjacency matrix apq 1 if theres an edge
  between p and q 0 otherwise.
• Matrix is V x V.
• Edge insertion/deletion/lookup O(1)
• Get set Ev of all edges incident on V? O(V)

14
Data Representations

• Adjacency lists
• For each node, store list of edges that meet it
• Insertion, lookup, deletion O(Ev)
• Getting edge list O(1)
• Surprisingly, this is preferred!

15
Exercise

• Draw adjacency matrix and adjancy lists for this
  graph

A
B
E
D
C
16
Depth First Search of a Graph

• Explore as far as possible from any node
• Only backtrack when necessary
• Knossos algorithm for exploring a maze
• Start with a ball of string and a piece of chalk
• Tie string at starting point
• At each junction, use chalk to mark the path
  youre exploring
• When you reach a cul-de-sac, backtrack until you
  find an unmarked path and take it

17
Dasgupta Code

• proc dfs(G)
• for all v in V
• visited(v) false // erase chalk marks!
• for all v in V
• if not visited(v) explore(G, v)
• proc explore(G, v)
• visited(v) true // make chalk mark
• previsit(v)
• for each edge (u, v) in E
• if not visited(u) explore(G, u)
• postvisit(v)

18
Observations

• The outer procedure makes sure that every part
  of the graph is explored (i.e., works even if
  graph is disconnected)
• The algorithm is O(V E)

19
Wheres the string?

• The ball of string is in the call stack
• Recursive call ? unwind some string
• Return from call ? wind up some string

20
Wikipedia Code

• def dfs(root, visited None,
• preorder_process lambda x None)
• """ Given a starting vertex, root, do a
  depth-first search.
• """
• import collections.deque
• to_visit collections.deque()
• if visited is None
• visited set()
• to_visit.append(root) Start with root
• while len(to_visit) ! 0
• v to_visit.pop()
• if v not in visited
• visited.add(v)
• preorder_process(v)
• to_visit.extend(v.neighbors)

21
Flash ActionScript code

• searchMethod function ()
• var originNode nodeOrigin
• var destinationNode nodeDestination
• var closedListArray new Array()
• var openListArray new Array()
• openList.push(origin)
•   while (openList.length ! 0)
• var nNode Node(openList.shift())
•   if (n destination)
• closedList.push(destination)
• trace("Closed!")
• break
•   var neighborsArray n.getNeighbors()
• var nLengthNumber neighbors.length
•   for (i0 i
• openList.unshift(neighborsnLength - i - 1)
• closedList.push(n)

22
Code enhancement (modified since lecture!)

• proc dfs(G)
• for all v in V
• visited(v) false // erase chalk marks!
• for all e in E
• status(e) NONE
• for all v in V
• if not visited(v) explore(G, v)
• proc explore(G, v)
• visited(v) true // make chalk mark
• previsit(v)
• for each edge e (u, v) in E
• if not visited(u)
• status(e) DISCOVERY
• explore(G, u)
• else
• if (status(e) NONE) status(e) BACK
• postvisit(v)

23
Example
24
Example
25
Observations

• The collection of all discovery edges forms a
  forest a collection of trees containing
  every node of the graph
• If the graph is connected, this is called a
  spanning tree

26
Finding a path to a target

• Previsit checks whether youve found target
• If not, push node onto a stack
• If so, report stack-contents as route from target
  to start-point
• Postvisit pop node off stack

27
Problem cycle finding

• Use DFS to either
• Find a cycle in a graph (report the nodes of the
  cycle), or
• Report that there are no cycles in the graph

28
Properties of DFS

• Define
• previsit(n)
• startTimen clock
• postvisit(b)
• finishTimen clock

29
Properties of DFS (2)

• Resulting intervals startn, endn are
  disjoint or contained never overlapping
• Child interval contained in parent interval

30
Observation

• The same DFS code works for directed graphs!

31

• FOUR kinds of edges
• Tree from node to a child
• Forward from node to non-child descendant
• Back from node to ancestor
• Cross neither to ancestor nor descendant to an
  already completely explored node

32

• FOUR kinds of edges
• Tree from node to a child
• Forward from node to non-child descendant
• Back from node to ancestor
• Cross neither to ancestor nor descendant to an
  already completely explored node

33
Breadth-first search

• DFS explores whole tree
• But it may take a long trip to discover a nearby
  node
• BFS makes sure to explore nearby nodes first
• At the start, explore only starting node
• i.e., all nodes whose edge distance from start
  is 0
• Then explore its neighbors
• i.e., all nodes whose edge distance from start is
  1
• Repeat
• Dont explicitly compute edge distance
• After exploring nodes at dist k, unexplored
  neighbors of the explored set are the dist k1
  nodes

34
BFS
35
Observations

• Only finds nodes reachable from the starting
  nodes for others, dist infinity
• Running time is O(V E)

36
Generalizations

• What if there are edge-lengths on each edge,
  instead of every edge being length one?
• Then we have the shortest path in a weighted
  graph problem
• Useful in path-planning for robots
• Cell-phone message forwarding
• Autonomous sensor clusters
• Will study this problem soon

37
Master Theorem

• Suppose that R is a function from positive
  integers to positive reals, and
• R(1) c
• R(n) f(n) a R(ceil(n/b)) for n 1
• where the function f is O(nd), a 0, b 1, and
  d ³ 0
• then
• if d
• if d logba, R is O(nd log n)
• if d logba, R is O(nd)

38
Master Theorem

• What does if d
• d 1)
• bd
• bd

39
Master Theorem

• Suppose that R is a function from positive
  integers to positive reals, and
• R(1) c
• R(n) f(n) a R(én/bù) for n 1
• where f(n) is O(nd), a 0, b 1, and d ³ 0
• then
• if a bd, R is O(n logb a)
• if a bd, R is O(nd log n)
• if a

40
Lemma

• Since for r different from 1,
• 1 r r2 rk-1 (rk 1)/(r-1),
• and for r 1,
• 1 r r2 rk-1 k
• we can say that
• 1 r r2 rk-1
• is
• O(rk-1) for r 1,
• O(k) for r 1, and
• O(1) for r

41
Proof

• Plug-n-chug!
• Well plug-n-chug k times, where k logbn
  thatll be enough to make the last term involve
  R(1)

42
Plug-n-chug

• R(n) f(n) a R(n/b)
• f(n) a f(n/b) aR(n/b2)
• f(n) a f(n/b) a2R(n/b2)
• f(n) a f(n/b) a2f (n/b2)
  aR(n/b3)
• f(n) a f(n/b) a2f (n/b2)
  a3R(n/b3)
• f(n) af(n/b) ak-1f(n/bk-1)
  akR(1)
• f(n) af(n/b) ak-1f(n/bk-1)
  akc

43
Plug-n-chug (2)

• R(n) f(n) af(n/b) ak-1f(n/bk-1) akc
• Because f(n) is O(nd), we can replace f(u) by tud
  as an overestimate
• R(n) f(n) af(n/b) ak-1f(n/bk-1) akc
• tnd a t (n/b)d ak-1 t (n/bk-1)d
  akc
• tnd1 a/bd ak-1(1/bk-1)d
  akc
• tnd1 a/bd ak-1(1/bd)k-1 akc
• tnd1 a/bd (a/bd)k-1
  akc
• tnd1 r rk-1 akc
• where r a/bd
• Note r

44
Plug-n-chug (3)

• R(n) tnd1 r rk-1 akc where
  r a/bd
• Case 1 r
• bracketed factor is O(1), so first term is O(nd)
• last term (akc) is c . a logbn c . nlogba
  cnd, so sum is O(nd)

45
Plug-n-chug (4)

• R(n) tnd1 r rk-1 akc
• where r a/bd
• Case 2 r 1.
• Bracketed factor is k, which is logb n
• First term is O(nd log n)
• Last term (akc) is c . a logbn c . nlogba
  cnd
• So entire expression is O(nd log n)
• Note logbn const . log(n).

46
Plug-n-chug (5)

• R(n) tnd1 r rk-1 akc
• where r a/bd
• Case 3 r 1, i.e., a bd, so logba d
• Bracketed factor is O(rk-1) treat as rk
• factor of r is just a constant
• rk rlogbn nlogbr nlogb(a) logb(bd)
  nlogb(a) d
• So first term is O(nlogba)
• Last term (akc) is c . a logbn c . nlogba
• So entire expression is O(nlogba)