Algorithm

1
Algorithm

• An algorithm is a sequence of unambiguous
  instructions for solving a problem, i.e., for
  obtaining a required output for any legitimate
  input in a finite amount of time.

2
Features of Algorithm

• Besides merely being a finite set of rules that
  gives a sequence of operations for solving a
  specific type of problem, an algorithm has the
  five important features Knuth1
• finiteness (otherwise computational method)
• termination
• definiteness
• precise definition of each step
• input (zero or more)
• output (one or more)
• effectiveness
• Its operations must all be sufficiently basic
  that they can in principle be done exactly and in
  a finite length of time by someone using pencil
  and paper Knuth1

3
Historical Perspective

• Muhammad ibn Musa al-Khwarizmi 9th century
  mathematician www.lib.virginia.edu/science/parshal
  l/khwariz.html
• Euclids algorithm for finding the greatest
  common divisor

4
Greatest Common Divisor (GCD) Algorithm 1

• Step 1 Assign the value of minm,n to t
• Step 2 Divide m by t. If the remainder of this
  division is 0, go to Step 3 otherwise, to Step
  4
• Step 3 Divide n by t. If the remainder of this
  division is 0, return the value of t as the
  answer and stop otherwise, proceed to Step 4
• Step 4 Decrease the value of t by 1. Go to Step
  2
• Note m and n are positive integers

5
Correctness

• The algorithm terminates, because t is decreased
  by 1 each time we go through step 4, 1 divides
  any integer, and t eventually will reach 1 unless
  the algorithm stops earlier.
• The algorithm is partially correct, because when
  it returns t as the answer, t is the minimum
  value that divides both m and n

6
GCD Procedure 2

• Step 1 Find the prime factors of m
• Step 2 Find the prime factors of n
• Step 3 Identify all the common factors in the two
  prime expansions found in Steps 1 and 2. If p is
  a common factor repeated i times in m and j times
  in n, assign to p the multiplicity mini,j
• Step 4 Compute the product of all the common
  factors with their multiplicities and return it
  as the GCD of m and n
• Note as written, the algorithm requires than m
  and n be integers greater than 1, since 1 is not
  a prime

7
Procedure 2 or Algorithm 2?

• Procedure 2 is not an algorithm unless we can
  provide an effective way to find prime factors of
  a number
• The sieve of Eratosthenes is an algorithm that
  provides such an effective procedure

8
Euclids Algorithm

• E0 Ensure m geq n If m lt n, exchange m with n
• E1 Find Remainder Divide m by n and let r be
  the remainder (We will have 0 leq r lt n)
• E2 Is it zero? If r0, the algorithm
  terminates n is the answer
• E3 Reduce Set m to n, n to r, and go back to E1

9
Termination of Euclids Algorithm

• The second number of the pair gets smaller with
  each iteration and cannot become negative
  indeed, the new value of n is r m mod n, which
  is always smaller than n. Eventually, r becomes
  zero, and the algorithms stops.

10
Correctness of Euclids Algorithm

• After step E1, we have m qn r, for some
  integer q. If r 0, then m is a multiple of n,
  and clearly in such a case n is the GCD of m and
  n. If r !0, note that any number that divides
  both m and n must divide m qn r, and any
  number that divides both n and r must divide qn
  r m so the set of divisors of m,n is the
  same as the set of divisors of n,r. In
  particular, the GCD of m,n is the same as the
  GCD of n,r. Therefore, E3 does not change the
  answer to the original problem. Knuth1

11
Algorithm Design Techniques

• Euclids algorithm is an example of a Decrease
  and Conquer algorithm it works by replacing its
  instance by a simpler one (in step E3)
• It can be shown (exercise 5.6b) that the instance
  size, when measured by the second parameter, n,
  is always reduced by at least a factor of 2 after
  two successive iterations of Euclids algorithm
• In Euclids original, the remainder m mod n is
  computed by repeated subtraction of n from m
• Step E0 is not necessary for correctness it
  improves time efficiency

12
Notion of algorithm
problem
algorithm
computer
input
output
Algorithmic solution
13
Example of computational problem sorting

• Statement of problem
• Input A sequence of n numbers lta1, a2, , angt
• Output A reordering of the input sequence lta1,
  a2, , angt so that ai aj whenever i lt j
• Instance The sequence lt5, 3, 2, 8, 3gt
• Algorithms
• Selection sort
• Insertion sort
• Merge sort
• (many others)

14
Selection Sort

• Input array a1,,an
• Output array a sorted in non-decreasing order
• Algorithm
• for i1 to n
• swap ai with smallest of ai1,an

• see also pseudocode, section 3.1

15
Sorting Some Terms

• Key
• Stable sorting
• In place sorting

16
Graphs Some Terms

• Graph
• Vertices, edges
• Undirected, directed
• Loops
• Complete, dense, sparse
• Paths sequences of vertices or of edges?
• Path length number of edges in path
• Walk repeated vertices and edges allowed
• Path repeated vertices are allowed, repeated
  edges are not allowed
• Simple path neither repeated vertices nor
  repeated edges are allowed
• Cycle simple path that starts and ends at the
  same vertex

17
Some Well-known Computational Problems

• Sorting
• Searching
• Shortest paths in a graph
• Minimum spanning tree
• Primality testing
• Traveling salesman problem
• Knapsack problem
• Chess
• Towers of Hanoi
• Program termination

18
Basic Issues Related to Algorithms

• How to design algorithms
• How to express algorithms
• Proving correctness
• Efficiency
• Theoretical analysis
• Empirical analysis
• Optimality

19
Algorithm design strategies
See http//www.aw-bc.com/info/levitin/taxonomy.htm
l This is not a unique taxonomy.

• Greedy approach
• Dynamic programming
• Backtracking and Branch and bound
• Space and time tradeoffs

• Brute force
• Divide and conquer
• Decrease and conquer
• Transform and conquer

20
Analysis of Algorithms

• How good is the algorithm?
• Correctness
• Time efficiency
• Space efficiency
• Does there exist a better algorithm?
• Lower bounds
• Optimality

21
What is an algorithm?

• Recipe, process, method, technique, procedure,
  routine, with the following requirements
• Finiteness
• terminates after a finite number of steps
• Definiteness
• rigorously and unambiguously specified
• Input
• valid inputs are clearly specified
• Output
• can be proved to produce the correct output given
  a valid input
• Effectiveness
• steps are sufficiently simple and basic

22
Why study algorithms?

• Theoretical importance
• the core of computer science
• Practical importance
• A practitioners toolkit of known algorithms
• Framework for designing and analyzing algorithms
  for new problems

23
Correctness

• Termination
• Well-founded sets find a quantity that is never
  negative and that always decreases as the
  algorithm is executed
• Partial Correctness
• For recursive algorithms induction
• For iterative algorithms axiomatic semantics,
  loop invariants

24
Complexity

• Space complexity
• Time complexity
• For iterative algorithms sums
• For recursive algorithms recurrence relations