Analysis of Algorithms Efficiency

1

• LECTURE 4
• Analysis of Algorithms Efficiency

2
Outline

• What is efficiency analysis ?
• How can be time efficiency measured ?
• Examples
• Best-case, worst-case and average-case analysis

3
What is efficiency analysis ?

• Analyzing the efficiency of an algorithm means
• To establish the amount of computing resources
  needed to execute the algorithm
• Remark it is sometimes called complexity
  analysis
• Usefulness it is useful to compare the
  algorithms

4
What is efficiency analysis ?

• Computing resources
• Memory space space needed to store the data
  processed by the algorithm
• Running time time needed to execute the
  operations of the algorithms
• Efficient algorithm an algorithm which uses a
  reasonable amount of computing
  resources
• If an algorithm uses less resources than another
  one then it is preferred

5
What is efficiency analysis ?

• There are two types of efficiency
• Space efficiency it refers to the space the
  algorithm requires
• How much space ?
• Time efficiency it refers to how fast an
  algorithm runs
• How much time ?
• Both analyses are based on the following remark
• the amount of computing resources depends on
  the dimension of processed data problems size
  input size
• Main aim of the analysis
• how depends the time and/or space
  on the input size ?

6
How can we determine the inputs size for a given
problem ?

• Usually, quite straightforward starting from
  the problem
• Inputs size dimension of the space needed to
  store all input data of the problem
• It can be expressed as
• the number of elements (real value, integer
  value, character etc) belonging to the input data
• the number of bits necessary to codify a value

7
How can we determine the inputs size for a given
problem ?

• Examples
• Find the minimum of an array x1..n
• Input size n
• Compute the value of a polynomial of order n
• Input size n
• Compute the sum of two matrices mn
• Input size (m,n) or mn
• Verify if a number n is prime or not
• Input size n or log2n1

8
What is efficiency analysis ?

• The efficiency is influenced also by the manner
  of organizing the data
• Example sparse matrix 100100 containing only
  50 non-zero elements
• Representation variants
• Classic (10010010000 values)
• One dimensional array of non-zero elements
• for each non-zero element
  (i,j,ai,j) (150 values)
• Second variant more efficient with respect to
  space
• First variant more efficient with respect to
  time
• We have to make a compromise between space
  efficiency and time efficiency

9
Outline

• What is efficiency analysis ?
• How can be time efficiency measured ?
• Examples
• Best-case, worst-case and average-case analysis

10
How can be time efficiency measured ?

• A measure for time efficiency
• estimate of running
  time
• To estimate the running time we must choose
• A computation model
• A measuring unit

11
How can be time efficiency measured ?

• A computation model random access machine (RAM)
• Characteristics
• All processing steps are sequentially executed
• (there is not
  parallelism in algorithms execution)
• The time of executing the basic operations does
  not depend on the values of the operands
• (there is no difference between
  computing 12 and 124334567)
• The time to access data does not depend on their
  address (there are no differences between
  processing the first element of an array and
  processing the last element)

12
How can be time efficiency measured ?

• Measuring unit time needed to execute a basic
  operation
• Basic operations
• Assignment
• Arithmetical operations
• Comparisons
• Logical operations
• Running time number of executions of the basic
  operations
• Remark. Running time expresses the dependence of
  the number of executed operations on the input
  size

13
Outline

• What is efficiency analysis ?
• How can be time efficiency measured ?
• Examples
• Best-case, worst-case and average-case analysis

14
Example 1

• Preconditions ngt1
  Postcondition S12n
• Inputs size n

Algorithm Operation Cost Repetitions 1 c1
1 2 c2 1 3 c3 n1 4 c4
n 5 c5 n ------------------------------------
-------- Running time T(n)(c3c4c5)n(c1
c2c3) an b
Algorithm Sum(n) 1 S0 2 i1 3 WHILE iltn
DO 4 SSi 5 ii1 6 ENDWHILE 7 RETURN S
15
Example 1

• Remarks
• Considering that all basic operations have an
  unitary cost one obtains T(n)3(n1)
• The values of the constants appearing in the
  expression of the running time are not very
  important. The fact which is important is that
  the running time depends linearly on the input
  size
• Since the algorithm is equivalent to
• S0
• FOR i1,n DO SSi ENDFOR
• it is easy to see that the cost of updating the
  counting variable i is
• 2(n1)

16
Example 2

• Preconditions Amn, Bnp
  Postcondition CAB
• Input size (m,n,p)

1
1
n
1
p
p
1
1
1
Bk,j, k1..n

x
Ci,j
Ai,k, k1..n
m
n
m
A
C
B
Ci,jAi,1B1,jAi,2B2,jAi,nBn,j
, i1..m, j1..p
17
Example 2

• Basic idea for each i1..m and j1..p compute
  the sum after k

Algorithm Product(A1..m,1..n,B1..n,1..p) 1
FOR i1,m DO 2 FOR j1,p DO 3
Ci,j0 4 FOR k1,n DO 5
Ci,jCi,jAi,kBk,j 6 ENDFOR 7
ENDFOR 8 ENDFOR 9 RETURN C1..m,1..p

• Costs table
• Op. Cost Rep Total
• 2(m1) 1 2(m1)
• 2(p1) m 2m(p1)
• 1 mp mp
• 2(n1) mp 2mp(n1)
• 2 mpn 2mnp
• -------------------------------------------
• T(m,n,p)4mnp5mp4m2

18
Example 2

• Remark is no need to do always such a detailed
  analysis
• it suffices to identify a dominant
  operation and to count it
• Dominant operation most frequent operation

Algorithm Product(A1..m,1..n,B1..n,1..p) 1
FOR i1,m DO 2 FOR j1,p DO 3
Ci,j0 4 FOR k1,n DO 5
Ci,jCi,jAi,kBk,j 6 ENDFOR 7
ENDFOR 8 ENDFOR RETURN C1..m,1..p
Analysis T(m,n,p)mnp
19
Example 3

• Preconditions x1..n, ngt1 Postconditions
  mmin(x1..n)
• Input size n

Algorithm Minimum(x1..n) 1 mx1 2 FOR
i2,n DO 3 IF xiltm THEN 4
mxi 5 ENDIF 6
ENDFOR 7RETURN m

• Table of costs
• Op. Cost Rep. Total
• 1 1 1
• 2n 1 2n
• 1 n-1 n-1
• 1 t(n) t(n)
• T(n)3nt(n)
• The running time depends not only on n but also
  on the properties of input data

20
Example 3

• When the running time depends also on the
  properties of input data
• we have to analyze two cases
• Best case (x1ltxi, i1..n) t(n)0 gt
  T(n)3n
• Worst case (x1gtx2gtgtxn) t(n)n-1gt
  T(n)4n-1
• Thus 3nltT(n)lt4n-1
• Both the lower and the upper bound
• depend linearly on the input size

Algorithm Minimum(x1..n) 1 mx1 2 FOR
i2,n DO 3 IF xiltm THEN 4
mxi 5 ENDIF 6 ENDFOR 7
RETURN m
Dominant operation
comparison T(n) n-1
21
Example 4

• Preconditions x1..n, ngt1, v a value
• Postconditions the variable found contains
  the truth value of the statement the value v is
  in the array x1..n
• Input size n

Algorithm (sequential search) search(x1..n,v) 1
found False 2 i1 3 WHILE
(foundFalse) AND (iltn) DO 4 IF xiv
//t1(n) 5
THEN found True //t2(n) 6
ELSE ii1 //t3(n) 7
ENDIF 8 ENDWHILE 9 RETURN found

• Costs table
• Op. Cost
• 1
• 1
• t1(n)1
• t1(n)
• t2(n)
• t3(n)

22
Example 4

• The running time depends on the properties of the
  array.
• Case 1 the value v is in the array (let k be
  the position where v is placed)
• Case 2 the value v is not in the array

k if v is in the array t1(n)
n if v is not in the array
1 if v is in the array t2(n)
0 if v is not in the array
k-1 if v is in the array t3(n) n
if v is not in the array
Algorithm (sequential search) search(x1..n,v) 1
found False 2 i1 3 WHILE
(foundFalse) AND (iltn) DO 4 IF xiv
// t1(n) 5
THEN found True // t2(n) 6
ELSE ii1 // t3(n) 7
ENDIF 8 ENDWHILE 9 RETURN found
23
Example 4

• Best case x1v
• t1(n)1, t2(n)1, t3(n)0
• T(n) 6
• Worst case v is not in the array
• t1(n)n, t2(n)0, t3(n)n
• T(n)3n3
• The lower and the upper bound
• 6lt T(n) lt 3(n1)
• The lower bound is constant, the upper bound
  depends linearly on n

k if v is in the array t1(n)
n if v is not in the array
1 if v is in the array t2(n)
0 if v is not in the array
k-1 if v is in the array t3(n) n
if v is not in the array
24
Example 4

• For some problems the best case and the worst
  case appear seldom
• Thus the running time in the best case and in
  the worst case do not give us enought information
• Another type of analysis average case analysis
• The aim of average case analysis is to give us
  information about the behavior of the algorithm
  for typical (random) input data

25
Outline

• What is efficiency analysis ?
• How can be time efficiency measured ?
• Examples
• Best-case, worst-case and average-case analysis

26
Best-case and worst-case analysis

• Best case analysis
• gives us a lower bound for the running time
• it can help us to identify algorithms which are
  not efficient (if an algorithm has a high cost in
  the best case it cannot be considered as a viable
  solution)

• Worst case analysis
• gives us the largest running time with respect to
  all input data of size n (this is an upper bound
  of the running time)
• the upper bound of the running time is more
  important than the lower bound

27
Average-case analysis

• This analysis is based on knowing the
  distribution probability of the input space.
• This means to know which is the appearance
  probability of each instance of input data (how
  frequently appears each instance of input data)
• The average running time is the mean value (in a
  statistical sense) of the running times
  corresponding to different instances of input
  data.

28
Average-case analysis

• Hypotheses Let us suppose that
• the input data can be grouped in classes such
  that for input data in the same class the running
  time is the same
• there are mM(n) such classes
• the probability of appearing an input data
  belonging to class k is Pk
• the running time of the algorithm for data
  belonging to class k is Tk(n)
• Then the average running time is
• Ta(n) P1T1(n)P2T2(n)PmTm(n)
• Remark if all classes have the same probability
  then
• Ta(n)(T1(n)T2(n)Tm(n
  ))/m

29
Average-case analysis

• Example sequential search (dominant
  operation comparison)
• Hypotheses concerning the probability
  distribution of input space
• Probability that the value v is in the array p
• - the value v appears with the same
  probability on any position of the array
• - the probability that the value v is on
  position k is 1/n
• Probability that the value v is not in the array
  1-p
• Ta(n)p(12n)/n(1-p)np(n1)/2(1-p)n(1-p/2)n
  p/2
• If p0.5 one obtains Ta(n)3/4 n1/4
• The average running time of sequential search is,
  as in the worst case, linear with respect to the
  input size

30
Average-case analysis

• Example sequential search (flag variant)
• Basic idea
• the array is extended with a position (n1) and
  on this position is placed v
• the extended array is searched until the value v
  is found (it will be found at least on position
  n1 in this case we can decide that the value
  is not in the initial array x1..n)

x1
x2
x3
xn
v
flag
31
Average-case analysis

• Algorithm
• Search_flag(x1..n,v)
• i1
• WHILE xiltgtv DO
• ii1
• ENDWHILE
• RETURN i
• Dominant operation comparison
• The probability that v is on position k is
  1/(n1)

• Average running time
• Ta(n)(12(n1))/(2(n1))
• (n2)/2
• Remark
• by changing the hypothesis on the distribution
  probability of input space the value of average
  running time changed (however it is still linear)

The average running time is NOT the arithmetical
mean of the running times corresponding to best
and average cases, respectively
32
Steps in estimating the running time

• Identify the input size
• Identify the dominant operation
• Count how many times the dominant operation is
  executed
• If this number depends on the properties of input
  data analyze
• Best case
• Worst case
• Average case

33
Next lecture will be

• also on analysis of algorithm efficiency.
• More specifically, we will discuss about
• growth order
• asymptotic notations
• complexity classes