Complexity

1

• Complexity

2
Complexity of Algorithms

• Obviously, on sorted sequences, binary search is
  more efficient than linear search.
• How can we analyze the efficiency of algorithms?
• We can measure the
• time (number of elementary computations) and
• space (number of memory cells) that the
  algorithm requires.
• These measures are called computational
  complexity and space complexity, respectively.

3
Complexity

• What is the time complexity of the linear search
  algorithm?
• We will determine the worst-case number of
  comparisons as a function of the number n of
  terms in the sequence.
• The worst case for the linear algorithm occurs
  when the element to be located is not included in
  the sequence.
• In that case, every item in the sequence is
  compared to the element to be located.

4
Linear Search

• Another example a linear search algorithm, that
  is, an algorithm that linearly searches a
  sequence for a particular element.
• procedure linear_search(x integer a1, a2, ,
  an integers)
• i 1
• while (i ? n and x ? ai)
• i i 1
• if i ? n then location i
• else location 0
• location is the subscript of the term that
  equals x, or is zero if x is not found

5
Complexity

• For n elements, the loop
• while (i ? n and x ? ai) i i 1
• is processed n times, requiring 2n comparisons.
• When it is entered for the (n1)th time, only the
  comparison i ? n is executed and terminates the
  loop.
• Finally, the comparison if i ? n then location
  iis executed, so all in all we have a
  worst-case time complexity of 2n 2.

6
Complexity

• What is the time complexity of the binary search
  algorithm?
• Again, we will determine the worst-case number of
  comparisons as a function of the number n of
  terms in the sequence.
• Let us assume there are n 2k elements in the
  list, which means that k log n.
• If n is not a power of 2, it can be considered
  part of a larger list, where 2k lt n lt 2k1.

7
Binary Search

• Procedure binary search(x integer, a1,...,an
  increasing integers)
• i 1 left endpoint of search interval
• j n right endpoint of search interval
• while (i lt j )
• m ?(i j )/2?
• if x gt am then i m 1
• else j m
• end
• if x ai then location i
• else location 0
• location is the subscript of the term that
  equals x location is 0 of x is not found

19
25
27
31
35
38
3
5
7
9
10
18
47
51
60
68
8
Complexity

• In the first cycle of the loop
• while (i lt j)
• begin
• m ?(i j)/2?
• if x gt am then i m 1
• else j m
• end
• the search interval is restricted to 2k-1
  elements, using two comparisons.

9
Complexity

• In the second cycle, the search interval is
  restricted to 2k-2 elements, again using two
  comparisons.
• This is repeated until there is only one (20)
  element left in the search interval.
• At this point 2k comparisons have been conducted.

10
Complexity

• Then, the comparison
• while (i lt j)
• exits the loop, and a final comparison
• if x ai then location i
• determines whether the element was found.
• Therefore, the overall time complexity of the
  binary search algorithm is 2k 2 2 ?log n? 2.

11
Complexity

• In general, we are not so much interested in the
  time and space complexity for small inputs.
• For example, while the difference in time
  complexity between linear and binary search is
  meaningless for a sequence with n 10, it is
  gigantic for n 230.

12
Complexity

• For example, let us assume two algorithms A and B
  that solve the same class of problems.
• The time complexity of A is 5,000n, the one for B
  is ?1.1n? for an input with n elements.

13
Complexity

• Comparison time complexity of algorithms A and B

Algorithm A
Algorithm B
Input Size
n
5,000n
?1.1n?
10
50,000
3
100
500,000
13,781
1,000
5,000,000
2.5?1041
1,000,000
5?109
4.8?1041392
14
Complexity

• This means that algorithm B cannot be used for
  large inputs, while running algorithm A is still
  feasible.
• So what is important is the growth of the
  complexity functions.
• The growth of time and space complexity with
  increasing input size n is a suitable measure for
  the comparison of algorithms.

15
Time Complexity

• Consider
• a 7
• x 3
• for i 1 to n
• y ax
• Complexity?

16
Time Complexity

• Consider
• a 7
• b 2
• c 1
• x 3
• for i 1 to n
• for j 1 to n
• y abbc(abc)/(ab) 4
• Complexity?

17
Time Complexity
Typical Complexities O(1) O(log n) O(n) O(n
log n) O(n2) O(n3) NP

• cg(n)

f(n)
n
k
f(n) is O(g(n)) iff positive constants c and k
exist such that f(n) lt cg(n) for all n, n gt k.
18
Time Complexity

• Assume an algorithm takes
• 3n 3 steps Then it is a O(n) Algorithm
• 5n2 5n 105 steps Then it is a O(n2)
  Algorithm
• 2n3 2n 10 steps Then it is a O(n3)
  Algorithm
• f(n) is O(g(n)) iff positive constants c and n0
  exist such that f(n) lt cg(n) for all n, n gt k.
• 3n 3 lt 4n for all n, n gt 3 so k
  3, c 4
• 5n2 5n 105 lt 10n2 for all n, n gt 6
  so k 6, c 10
• 2n3 2n 10 lt 3n3 for all n, n gt 3
  so k 3, c 3

19
Efficiency Classes
O(1) Constant O(log n) Logarithmic O(n)
Linear O(n log n) n log n O(n2)
Quadratic O(n3) Cubic O(2n)
Exponential O(n!) Factorial
20
Example Find Maximum Element in a Finite Sequence

• Procedure max(a1, a2,..., an integers)
• max a1
• for i 2 to n
• if max lt ai then max ai
• max is the largest element

a
6
5
7
1
10
8
1 2 3 4 5 6
21
Example Unique Elements in a Sequence

• //Checks whether all elements in array are
  distinct
• Procedure unique(a0, a1,..., an-1 integers)
• for i 0 to n 2
• for j i 1 to n - 1
• if ai aj then return false
• return true

a
6
5
7
1
10
8
1 2 3 4 5 6
22
Example Unique Elements in a Sequence

• //Checks whether all elements in array are
  distinct
• Procedure unique(a0, a1,..., an-1 integers)
• for i 0 to n 2
• for j i 1 to n - 1
• if ai aj then return false
• return true
• Worst Case?

a
6
5
7
1
10
8
1 2 3 4 5 6
23
Homework

• P. 151
• 3, 7, 9, 11, 12