3rd Edition: Chapter 1

1
Chapter 8More Computational Complexity The
Searching Problem

• 8.1 Lower Bounds for Searching by
• Comparisons of Keys
• 8.2 Interpolation Search
• 8.3 Searching in Trees
• 8.4 Hashing
• 8.5 The Selection Problem Introduction to
• Adversary Arguments

2
8.1 Lower Bounds for Searching Only by
Comparisons of Keys
Decision Tree of Binary Search
3
Decision Tree of Sequential Search
Validfor each possible outcome there is
a path from the root to a leaf that
reports that outcome
Prunedevery leaf is reachable
4
Lower Bounds for Worst-Case Behavior

• Lemma 8.1
• If n is the number of nodes in a binary tree and
• d is the depth, then d ? ?lg n? .
• Pf

5
Lower Bounds for Worst-Case Behavior

• Lemma 8.2
• To be a pruned, valid decision tree for searching
• n distinct keys for a key x, the binary tree
• consisting of the comparison nodes must contain
• at least n nodes .
• Pf

6
Lower Bounds for Worst-Case Behavior

• Theorem 8.1
• Any deterministic algorithm that searches for a
• key x in an array of n distinct keys only by
• comparisons of keys must in the worst case do at
• least ?lg n? 1 comparisons of keys .
• Pf

7
Lower Bounds for Average-Case Behavior

• Lemma 8.3
• The tree consisting of the comparison nodes in
• the pruned, valid decision tree corresponding to
• Binary Search is a nearly complete binary tree.
• Pf
• Nearly Complete Binary Treeit is complete down
  to a depth of d?1.

8
Analysis of Algorithm 2.1

• Node Distance
• The total number of nodes in the path from the
• root to that node.
• Total Node Distance (TND)
• The sum of the node distances to all nodes of a
• tree.

9
Analysis of Algorithm 2.1
Decision Tree of Binary Search
TND 1223333 17
10
Analysis of Algorithm 2.1
Assume n 2k?1 for some integer k, and x in the
array For n in general,
11
Analysis of Algorithm 2.1
Consider the case that x may not in the
array. There are 2n1 possibilities x less than
all, x greater than all, or between any two of
them. Assume each possibility is equally
probable, For n in general,
12
Lower Bounds for Average-Case Behavior

• Lemma 8.4
• The TND of a binary tree containing n nodes is
• equal to minTND(n) if and only if the tree is
• nearly complete.
• Pf
• P.S. min TND(n) is the minimum of the TND for
  binary trees containing n nodes

13
Lower Bounds for Average-Case Behavior

• Lemma 8.5
• Suppose that we are searching n keys, the search
• key x is in the array, and all array slots are
• equally probable. Then the average-case time
• complexity for Binary Search is given by
• minTND(n)/n .
• Pf

14
Lower Bounds for Worst-Case Behavior

• Theorem 8.2
• Any deterministic algorithms that search for a
  key x in
• an array of n distinct keys only by comparisons
  of keys,
• Binary Search is optimal in its average-case
  performance
• if we assume that x is in the array and that all
  array slots
• are equally probable. Therefore, under these
• assumptions, any such algorithm must on the
  average do
• at least approximately ?lg n? ?1 comparisons of
  keys .
• Pf

15
8.5 The Selection Problem
Objective Find the kth-largest (or
kth-smallest) key in a list of n keys. We assume
that the keys are in an unsorted array. In this
course, we will only discuss the problem for k
1, that is, find the largest or smallest one in
an unsorted array.
16
Finding the Largest Key
Algorithm 8.2 void find_largest (int n, const
keytype S, keytype large) index i
large S1 for (i 2 i lt n i)
if (Sigtlarge) large
Si
Time Complexity T(n) n?1
Winnerlargest key is the winner
of the tournament Losersmaller key is the loser
of the tournament
17
Finding the Largest Key

• Theorem 8.7
• Any deterministic algorithm that can find the
• largest of n keys in every possible input only
  by
• comparison of keys must in every case do at
• least n?1 comparisons of keys,
• Pf

18
Finding Smallest and Largest Keys
Algorithm 8.3 void find_both (int n, const
keytype S, keytype small,

keytype large) index i small S1
large S1 for (i 2 i lt n
i) if (Si lt small)
small Si else if (Si gt large)
large Si
Time Complexity W(n)
19
Finding Smallest and Largest Keys
Algorithm 8.4 void find_both2 (int n, const
keytype S, keytype small,

keytype large) index i if (S1 lt
S2) small S1 large
S2 else small S2
large S1 for (i 3 i lt n?1
i i 2) if (Si lt Si1 )
if (Si lt small )
small Si if (Si1 gt
large) large Si1
else if (Si1 lt small )
small Si1 if (Si gt large)
large Si
Time Complexity T(n)
20
Finding Smallest and Largest Keys
One can show that The Decision tree does
not work well for showing the Selection Problem.
We use another method, called an adversary
argument, to establish our lower bound.
21
Adversary Argument Method
The adversarys goal is to make an algorithm as
hard as possible. Example In a Pub or Bar, a
stranger ask you the tired question, Whats your
sign? To make this stale encounter more
exciting, you tell the stranger to guess your
sign. You simply want to force the stranger to
ask as many questions as possible. For example,
the stranger ask, Were you born in summer?,
Were you born in a month of 31 days? .... But
your answers must be consistent.
22
Adversary Argument Method
We use an adversary argument to obtain a lower
bound on the worst-case number of comparisons
needed to find both the largest and smallest
keys. State Description X The key
has not been involved in a comparison. L
The key has lost at least one comparison and
has never won. (one unit of
information) W The key has won at least
one comparison and has never
lost. (one unit of information) WL The
key has won at least one comparison and has
lost at least one. (two units of
information) The algorithm must learns (2n?2)
units of information.
23
Example 8.2 (Table 8.2)
24
Example 8.2 (Table 8.3)
For Algorithm 8.3
25
Finding Smallest and Largest Key

• Theorem 8.8
• Any deterministic algorithm that can find the
• largest of n keys in every possible input only
  by
• comparison of keys must in the worst case do at
• least the following comparisons of keys
• Pf