CSCE 210 Data Structures and Algorithms

1
CSCE 210Data Structures and Algorithms

• Prof. Amr Goneid
• AUC
• Part 13b. Sorting(2)
• (n log n) Algorithms

2
Sorting(2) (n log n) Algorithms

• General
• Heap Sort
• Merge Sort
• Quick Sort

3
1. General

• We examine here 3 advanced sorting
• algorithms
• Heap Sort (based on Priority Queues)
• Merge Sort (based on Divide Conquer)
• Quick Sort (based on Divide Conquer)
• All of these algorithms have a worst case
• complexity of O(n log n)

4
2. Heap Sort

• The heap sort is a sorting algorithm based on
  Priority Queues.
• The idea is to insert all array elements into a
  minimum heap, then remove the top of the heap one
  by one back into the array.

5
HeapSort Algorithm

• //To sort an array X of integers of N elements
• void heapsort(int X , int N)
• int i
• PQltintgt Heap(N)
• for (i 1 i lt N i) Heap.insert(Xi)
• for (i 1 i lt N i) Xi Heap.remove()

6
Analysis of HeapSort

• Worst case cost of insertion
• This happens when the data are in descending
  order.
• In this case, every new insertion will have to
  take the element all the way up to the root, i.e.
  O(h) operations. Since a complete tree has a
  height of O(log n) , the worst case cost of
  inserting (n) elements into a heap is O(n log n)

7
Analysis of HeapSort

• Worst case cost of removal
• It is now easy to see that the worst case cost
  of removal of an element is O(log n). Removing
  all elements from the heap will then cost O(n log
  n).
• Worst Case cost of HeapSort
• Therefore, the total worst case cost for
  heapsort is O(n log n)

8
Performance of Heap Sort

• The complexity of the HeapSort is O(n log n)
• In-Place Sort No (uses heap array)
• Stable Algorithm Yes
• This technique is satisfactory for medium to
  large data sets

9
3. MergeSort (a) Merging

• Definition
• Combine two or more sorted sequences of data
  into a single sorted sequence.
• Formal Definition
• The input is two sorted sequences,
• Aa1, ..., an and Bb1, ..., bm
• The output is a single sequence, merge(A,B),
  which is a sorted permutation of
• a1, ..., an, b1, ..., bm.

10
Merge Algorithm
array B

• Merge(A,p,q,r)
• copy Ap..r to Bp..r and set Ap..r to empty
• while (neither L1 nor L2 empty)
• compare first items of L1 L2
• remove smaller of the two from its list
• add to end of A
• concatenate remaining list to end of A
• return A

q
L1
p
r
q1
L2
11
Example
i
j

B
q
r
p
A
z
12
Example
i
j

B
q
r
p
A
z
13
Example
i
j

B
q
r
p
A
z
14
Example
i
j

B
q
r
p
A
z
15
Example
i
j

B
q
r
p
A
z
16
Example
i
j

B
q
r
p
A
z
17
Example
i
j

B
q
r
p
A
z
18
Example
i
j

B
q
r
p
A
z
19
Implementation

• void Merge (Type A ,int p,int q,int r)
• // Merge lists Ap..q and Aq1..r
• // B is a global auxiliary array
• int i,j,z,k
• for(kp kltr k) Bk Ak
• i p j q1 z p
• while ((i lt q) (j lt r))
• if (Bi lt Bj) Az Bi i
• else Az Bj j
• z

20
Implementation

• if (i lt q)
• for (ki kltq k) Az Bk
• else if (j lt r)
• for (kj kltr k) Az Bk

21
Worst Case Analysis

• L1 size of L1, L2 size of L2
• In the worst case L1 L2 n/2
• Both lists empty at about same time, so
  everything has to be compared.
• Each comparison adds one item to A so the worst
  case is
• T(n) A-1 L1L2-1 n-1 O(n)
• comparisons.

22
(b) MergeSort Methodology

• Invented by Von Neumann in 1945
• Recursive Divide-And-Conquer
• Divides the sequence into two subsequences of
  equal size, sorts the subsequences and then
  merges them into one sorted sequence
• Fast, but uses an extra space

23
Methodology (continued)

• Divide
• Divide n element sequence into two subsequences
  each of n/2 elements
• Conquer
• Sort the two sub-arrays recursively
• Combine
• Merge the two sorted subsequences to produce a
  sorted sequence of n elements

24
Algorithm

• void MergeSort (Type A , int p, int r)
• // Mergesort array A locations p..r
• int q
• if (p lt r) // if there are 2 or more elements
• q (pr)/2 // Divide in the middle
• // Conquer both
• MergeSort (A , p , q) MergeSort(A , q1 , r)
• Merge(A , p , q , r) // Combine solutions
• Invoke as MergeSort(a , 1 , n)

25
Merge Sort Demo

• http//www.cosc.canterbury.ac.nz/people/mukundan/d
  sal/MSort.html

26
Performance of MergeSort

• MergeSort divides the array to two halves at each
  level. So, the number of levels is O(log n)
• At each level, merge will cost O(n)
• Therefore, the complexity of MergeSort is
  O(n log n)
• In-Place Sort No (uses extra array)
• Stable Algorithm Yes
• This technique is satisfactory for large data
  sets

27
4. QuickSort(a) Methodology

• Invented by Tony Hoare in 1962
• Recursive Divide-And-Conquer
• Partitions array around a Pivot , then sorts
  parts independently
• Fast, in-place sorting

28
Methodology (continued)

• Divide
• Array ap..r is rearranged into two nonempty
  sub-arrays ap..q and aq1..r such that each
  element of the first is lt each element of the
  second ( the pivot index is computed as part of
  this process)
• Conquer
• Sort the two sub-arrays recursively
• Combine
• The sub-arrays are already sorted in place,
  i.e., ap..r is sorted.

29
(b) Algorithm

• void QuickSort (Type a , int p , int r)
• if (p lt r )
• int q partition(a , p , r)
• QuickSort(a , p ,q)
• QuickSort(a , q1 , r)
• Invoke as QuickSort(a , 1 , n)

30
Partitioning

• int partition(Type a , int p, int r)
• Type pivot ap // pivot is the first
  element
• int i p-1 int j r1
• while (true)
• do j-- while (aj gt pivot)
• do i while (ai lt pivot)
• if(i lt j) swap (ai , aj)
• else return j
• // j is the location of last element in left
  part

31
Example using 1st element as pivot
i
r
j
p
Pivot 1st element 8
32
Partitioning
Pivot 8
i
j
33
Partitioning
Pivot 8
i
j
34
Partitioning
Pivot 8
i
j
35
Partitioning
Pivot 8
i
j
36
Partitioning
Pivot 8
j
i
37
Partitioning
Pivot 8
i
j
38
Partitioning
Pivot 8
i
j
39
Partitioning
Pivot 8
i
j
40
Partitioning
Pivot 8
i
j
41
Partitioning
i
j
final j
42
Partitioning
q final j
p
r
Right
Left
43
Example (continued)

Final Array
44
Quick Sort Demo

• http//www.cosc.canterbury.ac.nz/people/mukundan/d
  sal/QSort.html

45
Performance of QuickSort

• Partitioning will cost O(n) at each level
• Best Case Pivot has the middle value
• Quicksort divides the array to two equal halves
  at each level. So, the number of levels is O(log
  n)
• Therefore, the complexity of QuickSort is
  O(n log n)
• Worst Case Pivot is the minimum (maximum) value
• The number of levels is O(n)
• Therefore, the complexity of QuickSort is
  O(n2)
• In-Place Sort Yes
• Stable Algorithm No
• This technique is satisfactory for large data sets

46
See Internet Animation Sites

• For Example
• http//www.cs.ubc.ca/spider/harrison/Java/sorting-
  demo.html
• http//www.cs.princeton.edu/ah/alg_anim/version1/
  QuickSort.html