Algorithms and Data Structures

1
Algorithms and Data Structures

• Lecture 7

2
Agenda

• Sorting algorithms
• Insertion sort
• Merge sort
• Counting sort

3
Sorting

• Input unsorted set of n numbers lta1, a1, angt
• Output sorted set of n numbers (permutation of
  original set ) lta1, a1, angt, where
  a1lta1lt lt an
• E.g. before sorting lt3, 2, 1, 4, 0gt, after
  sorting lt0, 1, 2, 3, 4gt

4
Sorting Insertion sort

• Convenient for short sequences
• Performs in place sorting - does not require
  additional memory (excluding memory for local
  variables)
• At any moment of time being sorted array consists
  of two parts sorted and unsorted (one of them
  may be empty)
• Initially, sorted part consists of one element
  (as one element sequence is always sorted)
• The first element of unsorted part is removed
  from the array and compared against the elements
  of sorted part in right to left order

5
Sorting Insertion sort

• During the comparison we are looking for a
  element, which key is less or equal to the key of
  current element
• Visited elements of sorted part are shifted
  rightwards (as they greater than the current key)
• When appropriate position is found the current
  element is inserted
• Size of sorted part increases by one, size of
  unsorted part decreases by one each time we move
  an element from unsorted part to the sorted one

6
Sorting Insertion sort
7
Sorting Insertion sort

• Algorithm starts with sorted array of size 1 and
  unsorted array of size n-1
• Algorithm stops when size of sorted array becomes
  n
• Insertion sort is T(n2)

8
Sorting Insertion sort
9
Sorting Merge sort

• Exploits recursive approach
• Input is divided into two parts, then sorting of
  each part is performed the same way (recursive
  call)
• Input division is performed until input size
  becomes 1
• 1-element array is already sorted
• Output is obtained merging two parts of array
  into the single one merging is performed in each
  subsequent recursive call
• Merging requires O(n) of additional memory
• Merge sort is T(n log2n)

10
Sorting Merge sort
11
Sorting Merge sort
12
Sorting Merge sort
13
Sorting Quick sort

• Exploits recursive approach
• Input is divided into two parts, then sorting of
  each part is performed the same way (recursive
  call)
• Input division is performed until input size
  becomes 1
• 1-element array is already sorted
• Quick sort performs in place sorting that means
  algorithm does not require additional memory

14
Sorting Quick sort

• The main operation of algorithm is so named
  partition
• Partition splits initial sequence p...r into
  two subsequences pq and q1r
• Partition rearranges elements of initial sequence
  so that any element from the first subsequence is
  always less or equal to any element from the
  second one
• It is important that q must be less than r, plt q
  ltr

15
Sorting Quick sort
16
Sorting Quick sort

• The idea behind partitioning is the following
• We have to choose so called waterline key
• Array elements with keys less than the
  waterline are moved to the left side elements
  with keys greater than the waterline to the
  right side
• Thus any left lt any right

17
Sorting Quick sort

• Initial array is looked through from two sides
  from the left and right
• If Left-right walking encounters element gt
  waterline it stops
• If right-left walking encounters element lt
  waterline it stops too
• If left- and right- sides are intersected
  procedure returns position of division
• Otherwise elements are exchanged and procedure
  continues

18
Sorting Quick sort
19
Sorting Quick sort
20
Sorting Quick sort

• In worst cases, if left and right parts of array
  differ greatly, algorithm is T(n2)
• In common cases, if left and right parts of the
  array are more-less equal, algorithm is T(n
  log2n)

21
Sorting

• All previously considered sorting algorithms
  belong to a set of so called comparison sort
  algorithms
• The main idea behind them is to compare and
  rearrange pairs of keys (or records)
• Another kind of algorithms is available

22
Sorting Counting sort

• Algorithm is applicable for sequences with
  positive integer keys
• It is assumed that set of keys is limited
  (finite)
• Algorithm consumes O(k) additional memory, where
  k is a maximal key value if k is a very large
  integer application of algorithm may be very
  memory consuming or ever impossible
• We will consider two variants of counting sort
  simple - for integers and modified -for records
  with integer keys

23
Sorting simple counting sort

• First of all we allocate counting array of size
  k
• Each cell of counting array is a counter of key
  values, e.g. cell 0 is a counter of keys with
  value 0 and so on
• Each counter must be initially set to 0
• Going through the being sorted array (source) we
  collect counters after this step counting
  array contains all the information needed to
  build sorted array (destination)
• Going through the counting array from the
  beginning we build destination array by adding
  key values so many times as they appear in
  counting array
• Obtained destination array is sorted source array

24
Sorting simple counting sort
25
Sorting Counting sort
26
Sorting simple counting sort

• Memory allocation is O(1)
• Resetting of counters is O(k)
• Collecting counters is O(n)
• Building of destination array is O(k)O(n)
• Totally O(1)2O(k)2O(n)O(kn)
• If k is fixed kcn, where c - some constant
• O(k)cO(n) and finally
• Counting sort is O(n)

27
Sorting modified counting sort

• If sorted array consists of some records with
  integer keys and we need to preserve original
  order of records with the same key values
  simple counting sort is not appropriate algorithm
• Preservation of original order of records with
  the same key values is a property of stability
• Algorithm of simple counting sort is not stable

28
Sorting modified counting sort
29
Sorting modified counting sort

• The main differences of modified counting sort
  algorithm are the following
• Each counter contains a sum of records which keys
  are less or equal to the key of current counter
  (previously counter contained a number of records
  with specified key, not sum)
• Counter value is also a position in a destination
  array of the last record with current key
• Building of destination array is performed by
  walking through the source array from the end
  (not counting array)

30
Sorting modified counting sort
31
Sorting Counting sort
32
Sorting Counting sort
33
Sorting Summary

• Insertion sort T(n2)
• Merge sort T(n log2n)
• Quick sort T(n log2n)
• Counting sort O(n)
• Heap sort O(n log2n)

34
Q A