Searching, Sorting, and Complexity Analysis

1
Searching, Sorting, and Complexity Analysis

• (Chapter 4 in the textbook)

2
Things to Desire in a Program

• Correctness
• Robustness
• Maintainability
• Efficiency

3
Measuring Efficiency

• Empirical - use clock to get actual running times
  for different inputs
• Problems
• Different machines have different running times
• Some running times are so long that it is
  impractical to check them

4
Measuring Efficiency

• Analytical - use pencil and paper to determine
  the abstract amount of work that a program does
  for different inputs
• Advantages
• Machine independent
• Can predict running times of programs that are
  impractical to run

5
Complexity Analysis

• Pick an instruction that will run most often in
  the code
• Determine the number of times this instruction
  will be executed as a function of the size of the
  input data
• Focus on abstract units of work, not actual
  running time

6
Example Search for Minimum
int min(int a) int min Integer.MAX_VALUE
for (int i a) min Math.min(min, i)
return min
We focus on the assignment () inside the loop
and ignore the other instructions. for an array
of length 1, one assignment for an array of
length 2, 2 assignments . for an array of length
n, n assignments
7
Big-O Notation

• Big-O notation expresses the amount of work a
  program does as a function of the size of the
  input
• O(n) stands for order of magnitude n, or order of
  n for short
• Search for the minimum is O(n), where n is the
  size of the input (a list, a number, etc.)

8
Common Orders of Magnitude
Constant O(k) Logarithmic O(log2n) Linear O(
n) Quadratic O(n2) Exponential O(kn)
9
Common Orders of Magnitude
n O(log2n) O(n) O(n2) O(2n) 2
1 2 4 4 4
2 4 16 64 8 3
8 64 256 16 4 16
256 65536 32 5 32
1024 4294967296 64 6 64
4096 19 digits 128 7 128
16384 whoa! 256 8 256
65536 512 9 512 262144 1024 10
1024 1048576
10
Approximations

• Suppose an algorithm requires exactly 3N 3
  steps in the worst case
• As N gets very large, the difference between N
  and N K becomes negligible (where K is a
  constant)
• As N gets very large, the difference between N
  and N / K or N K also becomes negligible
• Use the highest degree term in a polynomial and
  drop the others (N2 N)/2 ? N2

11
Example Approximations
n O(n) O(n) 2 O(n2) O(n2) n 2
2 4 4 6 4
4 6 16 20 8 8
10 64 72 16 16 18
256 272 32 32 34 1024
1056 64 64 66 4096 5050
128 128 130 16384 16512 256 256
258 65536 65792 512 512 514
262144 262656 1024 1024 1026 1048576
1049600
12
Example Sequential Search
int find(int a, int target) for (int i 0
i lt a.length i) if (ai target)
return i // found target return
-1 // target not there
Which instruction do we pick? How fast is its
rate of growth as a function of n? Is there a
worst case and a best case? An average case?
13
Improving Search

• Assume data are in ascending order
• Goto midpoint and look there
• Otherwise, repeat the search to left or to right
  of midpoint

89
target
34
41
56
63
72
89
95
0 1 2 3 4 5 6
3
midpoint
left
right
14
Improving Search

• Assume data are in ascending order
• Goto midpoint and look there
• Otherwise, repeat the search to left or to right
  of midpoint

89
target
34
41
56
63
72
89
95
0 1 2 3 4 5 6
5
midpoint
right
left
15
Example Binary Search
int find(int a, int target) int first 0
int last a.length - 1 while (first lt
last) int mid (first last) / 2 //
compute midpoint if (amid target)
return mid // found target
else if (amid gt target) last
mid - 1 // search left half
else first mid 1 //
search right half return -1
// target not there
16
Analysis
while (first lt last) int mid (first
last) / 2 // compute midpoint if (amid
target) return mid //
found target else if (amid gt target)
last mid - 1 // search left
half else first mid 1
// search right half
How many times will be executed in the worst
case?
17
Sorting an Array
89
56
63
72
41
34
95
0 1 2 3 4 5 6
sort
34
41
56
63
72
89
95
0 1 2 3 4 5 6
18
Selection Sort

• For each position i in the array
• Select the smallest element from i to n - 1
• Swap it with the ith one

19
Trace
i
Step 1 find the smallest element
89
56
63
72
41
34
95
0 1 2 3 4 5 6
smallest
i
Step 2 swap with first element
34
56
63
72
41
89
95
0 1 2 3 4 5 6
i
Step 3 advance i and goto step 1
34
56
63
72
41
89
95
0 1 2 3 4 5 6
20
Design of Selection Sort
for each i from 0 to n - 1 minIndex
min (array, i, n) if (minIndex ! i)
swap (array, i, minIndex)

• The method min returns the index of the smallest
  element
• The method swap exchanges the elements at the
• specified positions

21
Implementation
private void selectionSort (int a) int
minIndex 0 for (int i 0 i lt a.length -
1 i) minIndex min (a, i, a.length)
if (minIndex ! i) swap (a, i,
minIndex)
22
Analysis of Selection Sort

• The main loop runs approximately n times
• Thus, the method min runs n times
• Within method min, a loop runs n - i times

23
Analysis of Selection Sort
Overall, the number of comparisons performed in
method min is n - 1 n - 2 n - 3 .
. 1 (n2 n) / 2
? n2