Dynamic Sets and Data Structures

1
Dynamic Sets and Data Structures

• Over the course of an algorithms execution, an
  algorithm may maintain a dynamic set of objects
• The algorithm will perform operations on this set
• Queries
• Modifying operations
• We must choose a data structure to implement the
  dynamic set efficiently
• The correct data structure to choose is based
  on
• Which operations need to be supported
• How frequently each operation will be executed

2
Some Example Operations

• Notation
• S is the data structure
• k is the key of the item
• x is a pointer to the item
• Search(S,k) returns pointer to item
• Insert(S,x)
• Delete(S,x) note we are given a pointer to item
• Minimum or Maximum(S) returns pointer
• Decrease-key(S,x)
• Successor or Predecessor (S,x) returns pointer
• Merge(S1,S2)

3
Basic Data Structures/Containers

• Unsorted Arrays
• Sorted Array
• Unsorted linked list
• Sorted linked list
• Stack
• Queue
• Heap

4
Puzzles

• How can I implement a queue with two stacks?
• Running time of enqueue?
• Dequeue?
• How can I implement two stacks in one array
  A1..n so that neither stack overflows unless
  the total number of elements in both stacks
  exceeds n?

5
Unsorted Array Sorted Array Unsorted LL Sorted LL Heap
Search
Insert
Delete
Max/Min
Pred/Succ
Merge
6
Case Study Dictionary

• Search(S,k)
• Insert(S,x)
• Delete(S,x)
• Is any one of the data structures listed so far
  always the best for implementing a dictionary?
• Under what conditions, if any, would each be
  best?
• What other standard data structure is often used
  for a dictionary?

7
Case Study Priority Queue

• Insert(S,x)
• Max(S)
• Delete-max(S)
• Decrease-key(S,x)
• Which data structure seen so far is typically
  best for implementing a priority queue and why?

8
Case Study Minimum Spanning Trees

• Input
• Weighted, connected undirected graph G(V,E)
• Weight (length) function w on each edge e in E
• Task
• Compute a spanning tree of G of minimum total
  weight
• Spanning tree
• If there are n nodes in G, a spanning tree
  consists of n-1 edges such that no cycles are
  formed

9
Prims algorithm

• A greedy approach to edge selection
• Initialize connected component N to be any node v
• Select the minimum weight edge connecting N to
  V-N
• Update N and repeat
• Dynamic set in Prims algorithm
• An item is a node in V-N
• The value of a node is its minimum distance to
  any node in N
• A minimum weight edge connecting N to V-N
  corresponds to the node with minimum value in V-N
  (Extract minimum)
• When v is added to N, we need to update the value
  of the neighbors of v in V-N if they are closer
  to v than other nodes in N (Decrease key)

10
Illustration

• Maintain dynamic set of nodes in V-N
• If we started with node D, N is now C,D
• Dynamic set values of other nodes
• A, E, F infinity
• B 4
• G 6
• Extract-min Node B is added next to N

11
Updating Dynamic Set

• Node B is added to N edge (B,C) is added to T
• Need to update dynamic set values of A, E, F
• Decrease-key operation
• Dynamic set values of other nodes
• A 1
• E 2
• F 5
• G 6
• Extract-min Node A is added next to N

12
Updating Dynamic Set Again

• Node A is added to N edge (A,B) is added to T
• Need to update dynamic set values of E
• Decrease-key operation
• Dynamic set values of other nodes
• E 2 (unchanged because 2 is smaller than 3)
• F 5
• G 6

1
4
2
A
B
C
D
5
6
3
10
2
5
E
F
G
13
Dynamic Set Analysis

• How many objects in initial dynamic set
  representation of V-N?
• How many extract-min operations need to happen?
• How many decrease-key operations may occur?
• Given all of the above, choose a data structure
  and tell me the implementation cost.
• Time to build initial dynamic set
• Time to implement all extract-min operations
• Time to implement all decrease-key operations

14
Kruskals Algorithm

• A greedy approach to edge selection
• Initialize tree T to have no edges
• Iterate through the edges starting with the
  minimum weight one
• Add the edge (u,v) to tree T if this does not
  create a cycle

15
Example

• (A,B)
• (A,E)
• (B,E) cycle
• (B,C)
• (F,G)
• (C,G)
• (B,F) cycle
• (C,D)
• (D,G) cycle

16
Disjoint Set Data Structure

• Given a universe U of objects (nodes V)
• Maintain a collection of disjoint sets Si that
  partition U
• Find-set(x) Returns set Si that contains x
• Merge(Si, Sj) Returns new set Sk Si union Sj
• Disjoint Sets and Kruskals algorithm
• Universe U is the set of vertices V
• The sets are the current connected components
• When an edge (u,v) is considered, we check for a
  cycle by determining if u and v belong to the
  same set
• 2 calls to Find-set(x)
• If we add (u,v) to T, we need to merge the 2 sets
  represented by u and v.
• Merge(Su,Sv)

17
Analysis

• How do we initialize the universe?
• How many calls to find-set do we perform?
• How many calls to merge-set do we perform?

18
Better data structures

• We need mergeable data structures that still
  support fast searches
• Binomial heaps (ch. 19)
• Fibonacci heaps (ch. 20)
• Disjoint set data structures (ch. 21)
• linked lists
• forests

19
Disjoint-set forests

• Representation
• Each set is represented as a tree, nodes point to
  parent
• Root element is the representative for the set,
  points to self or has null parent pointer
• Height maintain height of tree as an integer
• Operations
• Makeset make a tree with one node
• Find progress from current element to root
  element following links
• Union connect root of lower height tree to point
  to root of larger height tree
• Figures copied from Jeff Erickson, UIUC

20
Naïve implementation
Figure copied from Jeff Ericksons slides at UIUC.
21
union-by-rank or union-by-depth
Leads to height of any tree of n nodes being at
most O(lg n).
Figure copied from Jeff Ericksons slides at UIUC.
22
Path Compression
Leads to amortized cost of a(n), the inverse
ackerman function. For all practical purposes,
a(n) 4.
Figure copied from Jeff Ericksons slides at UIUC.
23
Binomial Heaps

• Binomial Tree
• Binomial Heap
• Figures copied from Dan Gildea, University of
  Rochester

24
Key idea Union in O(lg n) time
25
Binomial Trees
Tree Bk has 2k nodes. Bk has height k. Children
of the root of Bk are Bk-1, Bk-2, , B0 from left
to right. Max degree of an n-node binomial tree
is lg n.
26
Binomial Heap

• A binomial heap of n-elements is a collection of
  binomial trees with the following properties
• Each binomial tree is heap-ordered (parent is
  less than all children)
• No two binomial trees in the collection have the
  same size
• Number of trees will be O(lg n)

27
Example Binomial Heap
Binomial heap of 29 elements 29 11101 in binary.
28
Minimum Operation
Where does the minimum have to be? How can we
find minimum in general? Running time?
29
Union of 2 Binomial Heaps