Not all algorithms are created equally

1
Not all algorithms are created equally

• Insertion of words from a dictionary into a
  sorted list takes a very long time.
• Insertion of the same words into a trie takes
  very little time.
• How can we express this difference?

2
Empirical studies

• We can gather empirical data by running both
  algorithms on different data sets and comparing
  their performance.

3
Issue 1

• Empirical data reflects performance on individual
  cases the more data points, the better, but no
  general understanding of algorithm performance is
  gained

4
Issue 2

• You must code the algorithms to be compared.
  This can be a non-trivial task.

5
Issue 3

• Empirical performance measures depend on many
  factors
• implementation language
• compiler
• execution hardware

6
Desiderata(things desired as essential on-line
Webster)

• We want a measure of algorithm performance which
• gives performance bounds as problem size grows
  large,
• is implementation independent,
• describes performance of algorithm in the general
  case, not just specific cases, and
• allows performance of different algorithms to be
  compared.

7
Asymptotic notation

• There are many flavors of asymptotic notation.
  We will study one the big-Oh notation.
• big-Oh gives an upper bound
• typically used to express an upper-bound on the
  worst-case performance of an algorithm

8
Definition

• Given two functions, f and g, mapping natural
  numbers into non-negative reals, we say that f(n)
  O(g(n)) if there exist positive constants c and
  n0, such that
• f(n) lt c g(n), for all n gt n0

9
What does this mean?

• It means that f cant grow faster than g.
• Were interested only in what happens when the
  input size of the problem is large.
• g(n) is a bound on the running time of an
  algorithm whose actual (unknown) runtime is f(n).
• We guarantee that the time required by the
  algorithm grows no more quickly than g, as the
  problem size gets large.

10
Some Comparative Bounds

• expression name
• O(1) constant 
• O(log n) logarithmic 
• O(log2 n) log squared 
• O(n) linear 
• O(n log n) n log n
• O(n2) quadratic 
• O(n3) cubic 
• O(2n) exponential
• O(n!) factorial 

11
Basic examples
12
A little tougher

• If you know that f(n) O(n3), what can you say
  about f(n) O(n2)? Is it impossible, possible,
  or necessary?

13
A little tougher

• If you know that f(n) O(n3), what can you say
  about f(n) O(n2)? Is it impossible, possible,
  or necessary?
• It is possible.

14
How about

• If you know that f(n) O(n3), what can you say
  about f(n) O(n4)? Is it impossible, possible,
  or necessary?

15
How about

• If you know that f(n) O(n3), what can you say
  about f(n) O(n4)? Is it impossible, possible,
  or necessary?
• It is necessary.

16
One last example

• If you know that f(n) O(n3), what can you say
  about f(n) n4? Is it impossible, possible, or
  necessary?

17
One last example

• If you know that f(n) O(n3), what can you say
  about f(n) n4? Is it impossible, possible, or
  necessary?
• It is impossible.

18
Conventions

• First, it is common practice when writing big oh
  expressions to drop all but the most significant
  terms. Thus, instead of O(n2 n log n n) we
  simply write O(n2).
• Second, it is common practice to drop constant
  coefficients. Thus, instead of O(3n2), we simply
  write O(n2). As a special case of this rule, if
  the function is a constant, instead of, say
  O(1024), we simply write O(1).
• Of course, in order for a particular big oh
  expression to be the most useful, we prefer to
  find a tight asymptotic bound. For example,
  while it is not wrong to write f(n) n O(n3),
  we prefer to write f(n)O(n), which is a tight
  bound.

19
Determining Bounds

• How do we determine the bounds for a particular
  method?
• Examine the iterative or recursive portion to see
  how it interacts with the dataset.

20
Example

• public void foo (int array)
• for (int i 0 i lt array.size() i)
• System.out.println(arrayi)