DESIGN

1
DESIGN ANALYSIS OF ALGORITHMS
Evelyne Tropper SUNY Plattsbugh
2
1. IntroductionWhy data structures algorithms?
What are the main problems we face in
programming?

• Solving complex problem quickly
• Finding efficient ways to solve problems in
  minimal time
• Using least memory possible for large problems
• Finding generic code components to re-use

3
... 100,000 items ...

• How do I retrieve 1st, last, kth item?
• How do I insert 1st, last, kth item?
• How do I remove 1st, last, kth item?
• How do I find 1st, last, kth item?
• Depends on data structure
• How quickly can I do operations?
• Depends on data structure algoritms
• How do I evaluate speed of operation?
• Depends on DS, algo. complexity

4
What influences the time of execution of a
program?

• Speed of the machine
• Quality of the compiler
• Size of the input
• Quality of the programming
• types of data structures used
• efficiency of algorithms used

5
Relation of size of input time of execution of
program

• 10 input items execute in 10 nsec, 100 items in
  100 nsec. Linear complexity.
• 10 input items execute in 10 nsec, 100 items in
  10000 nsec. Quadratic complexity.
• 10 input items execute in 10 nsec, 100 items in
  210 nsec. Exponential complexity.
• 10 input items execute in 10 nsec, 100 items in
  log 100 nsec. Logarithmic complexity.

6
Complexity of algorithmrate of growth of
execution time
Which of the curves we obtain depends on the
COMPLEXITY of the algorithm.
7
To improve quality of programming

• Concentrate on problem (algorithm) rather than on
  the coding.
• Use a language that facilitates above.
• Find algorithms that take less time, using
  complexity analysis.
• Use efficient data structures.

8
How to do that?

• Separate the structure of the solution from its
  implementation.
• Use Object Based Programming (OBP) or Object
  Oriented Programming (OOP)
• Determine the complexity of the algorithm how
  to reduce it.
• Find data structures to do algorithms more
  efficiently.

9
Re-use of components
Re-usable component
Good complexity
10
Some typical problems

• Finding a name file of a client.
• Finding the largest order this month.
• Finding the latest version of a piece of code.
• Finding the books that have been out longest.

General manipulation of Data Bases
complex files.
11
Generic algorithms found inmost programs

• Searching
• Sorting organizing data
• Insertions in files or DBs
• Deletions
• Cutting complex problems into smaller, easier
  pieces

12
Advantages of efficient data structures

• A good representation of the data is easy to
  debug.
• A good representation is re-usable in other
  programs.
• A good representation facilitates efficient
  algorithms.
• A good representation can diminish complexity of
  algorithm time of execution.

13
5. Review of Complexity Analysis
The speed of execution depends on 4 things

• Speed of the machine
• Quality of the compiler
• Size of the input
• Quality of the algorithms
• types of data structures used
• efficiency of algorithms used

14
Complexity of algorithmrate of growth of
execution time
Which of the curves we obtain depends on the
COMPLEXITY of the algorithm.
15
Exec time vs. complexity
16
Examples

• O(n) is linear
• O(n2) is quadratic
• O(2n) is exponential
• O(nlogn) is logarithmic

17
Applications
For (int i 0 i lt A i) For
(int i 0 i lt B i)
Complexity is O(max(A, B)) which is linear.
For (int i 0 i lt max i) For (int j 0 j
lt max j)
Complexity is O(max2) which is quadratic.
18
How many bits are required to represent N
consecutive integers ?
Since B bits can represent 2B integers, to
represent N integers 2B N --gt B log N
Complexity is logarithmic.
19
Repeated doubling
From x 1, how many times should it be doubled
before x ? N?
After k doubling we have 2k. So we find the
smallest k such that 2k ? N.
Therefore k ? log2 N
The complexity is logarithmic.
20
Repeated halving
From x N, how many times should it be halved
before x ? 1? After k doubling we have (½)k. So
we find the smallest k such that (½) k ?
N. Therefore k ? log.5 N Same as repeated
doubling.
Complexity is O(log N)
21
Static searching problem

• Searching problem
• Return the position of integer X in an array
  A.
• Return a message if not present.
• Return one position if multiple occurrences.

The solution depends on whether the array is
sorted or not.
22
Sequential search

• An unsuccessful search requires examining every
  item in the array, so the complexity is O(N).
• For the worst case, we also examine every item in
  the array, so the complexity is O(N).
• On average, we examine N/2 items, so the
  complexity is still O(N/2) O(N).

23
Binary searches for sorted array
Not found item low
Not found item low
This comes down to the repeated halving principle
the complexity is O(log N) for all cases. This
is very fast! An unsuccessful search adds 1
iteration, an average search subtracts 1
iteration.
24
/ Binary search algorithm. Return index of
found item / public static int binarySearch(
Comparable A , Comparable x) throws
ItemNotFound int low 0 int high a.length
- 1 int mid while( low lt high) mid (
low high) / 2 if ( amid.compares (x) lt 0
) low mid 1 else if ( amid.compares
(x) gt 0 ) high mid - 1 else return
mid throw new ItemNotFound ( Binary
Search fails )
25
Interpolation searchFinding a better guess than
the middle

• Looking for B... in a personal telephone
  directory with 52 pages.
• Instead of looking at the middle first, we might
  try a better guess.
• 52/26 2 pages per letter of alphabet A
  should start on 1st page and B on the 3rd page.
• Length of directory, dir, is high - low 1
• Span of letters is dirhigh - dirlow
• Any letter H is H - dirlow away from
  beginning

26
H is not around ½ point but at H -
dirlow point.
dirhigh - dirlow of the total length.
So instead of a mid-point at ½ (high - low
1) we will have a next point with ½ replaced by
above fraction.
27
Formula for Interpolation Search
If each access is expensive (ex. disc access),
if data is uniformly distributed, interpolation
search is faster than a binary search. Mid is
replaced by next.
next low X - a low
(high - low 1) ahigh - alow
Worst case complexity is O(N). Average case is
O(log log N). For N 4x109, log N 32, log log
N 5
28
Limitations of complexity analysis

• For small amount of input, use simplest
  algorithm.
• A running time of 2N log N might be better than
  1000N although its growth rate is higher. The
  differences in memory disc accesses are
  important.
• Average cases are hard to define hard to
  obtain. Worst cases are usually used instead.

29
6. API collections
30
7. RECURSION

• Some examples are
• the computations of n!,
• dictionary searches,
• finding the sum of the 1st n integers,
• the Fibonacci numbers,
• binary searches,
• fractals,
• printing each digit in a number, etc.
• Recursion should only be used if the cost in time
  memory does not exceed that of the iterative
  algorithm .

31
Idea behind recursion

• n! n(n-1)(n-2) (3)(2)(1)
• If 1! 1 then 2! 2.1!
• then 3! 3.2!
• then 4! 4.3!

• So with 1! 1, all the others are n! n
  (n-1)!

• If n 1 then factorial(n) 1
• else factorial(n) n factorial(n-1)

32
// Factorial function using recursion public
static long fact( int n) if (n
1) return 1 else return (n fact(n-1))
33
A trace

• Let input n 5.
• n ? 1 return (5 fact(4))
• n ? 1 return (4 fact(3))
• n ? 1 return (3 fact(2))
• n ? 1 return (2 fact(1))
• n 1 return fact(1) 1
• evaluate fact(2) 2 1 2
• evaluate fact(3) 3 2 6
• evaluate fact(4) 4 6 24
• evaluate fact(5) 5 24 120

34

The recursion stack
n ? 1
Store 5 24 pop top
Store 4 6 pop top
Store 3 2 pop top
Store 2 1 pop top
Store 1 pop top
35
Example ?1n k
// Sum function using recursion public static
long sum( int n) if (n 1) return
1 else return (n sum(n-1))
36
A trace

• Let input n 5.
• n ? 1 return (5 sum(4))
• n ? 1 return (4 sum(3))
• n ? 1 return (3 sum(2))
• n ? 1 return (2 sum(1))
• n 1 return sum(1) 1
• evaluate sum(2) 2 1 3
• evaluate sum(3) 3 3 6
• evaluate sum(4) 4 6 10
• evaluate sum(5) 5 10 15

37
The iterative version
// Sum function using iteration public static
long sum( int n) int sum 0 for (i 1 i
lt n i ) sum sum i
38
A trace
sum lt-- 0 sum lt-- 0 1 1 sum lt-- 1
2 3 sum lt-- 3 3 6 sum
lt-- 6 4 10 sum lt-- 10 5 15
This is much shorter than the recursive method.
39
Example the Fibonacci numbers (sum of the last
2) 1, 1, 11, 12, 23, 35, 58, 813, The
formula is fib (n) fib (n-1) fib (n-2)
// The fibonacci sequence Public static long
fib(int n) if (n lt 1) return 1
else return fib(n-1) fib(n-2)
40
A trace
fib(5)
41
Example printing numbers in any base
If we want to print each digit of 1234 we use
recursion and n10 which gives us n (mod 10)
or n for n lt 10. Following is a routine for base
10.
// Print each digit of a decimal number n public
static printRoutine( int n) if ( n gt
10) printRoutine( n/10) printDigit (n10)
42
We could write a simplistic routine to handle any
base from 0-16
//Print the digits of a number in any base final
static String digitTable 0123456789abcdef pub
lic static void printInt ( int n, int base) if
( n gt base) printInt (n / base,
base) System.out.print( digitTable.charAt( n
base))
Note the problems we encounter for base 0, 1 or
gt16
43
// Driver routine public static void printInt(
int n, int base) if ( base lt 1 base gt
maxbase ) System.err.println( Illegal base
base) else if ( n lt 0 ) n -
n System.out.print( - ) recPrint(
n, base )
44
// Print the digits of a number in any
base static.final String digitTable
0123456789abcdef private static final int
maxBase digitTable.length() // recursive
routine public static void recPrint( int n, int
base) if ( n gt base) recPrint (n / base,
base) System.out.print( digitTable.charAt( n
base))
45
Binary search as a recursive routine
/ A recursive algorithm for Binary Search of the
same complexity as the iterative one. public
static int binSearch( Comparable a ,
Comparable x) throws ItemNotFound return
binSearch( a, x, 0, a.length - 1)
46
// Hidden recursive routine private static int
binSearch( Comparable a , Comparable x, int
low, int high, ) throws ItemNotFound
if ( low gt high)
throw new
ItemNotFound( Binary search failed) int
mid ( low high ) / 2 if (
amid.Compares(x) lt 0) return binSearch( a, x,
mid1, high) else if ( amid.Compares(x) gt
0) return binSearch( a, x, low, mid-1)
else return mid
47
Complexity of the recursive algorithm
Each recursive call looks at ½ of the remaining
array. It will take r times to exhaust the
n-array. So 2r n ? r log n
Once element has been found, or array has been
exhausted, it need not go back to the first item
put in the stack.
48
Drawing a ruler by halves recursively
Private void drawRuler( Graphics g, int left, int
right, int level) if (level lt
1) return int mid (left right) /
2 g.drawLine (mid, 80, mid, 80 - level
5) drawRuler ( g, left, mid -1, level
-1) drawRuler ( g, mid 1, right, level -1)
49
Dynamic Programming

• Some problems are recursive by nature, but the
  recursive algorithm is inefficient Fibonacci
  no.s
• A coin-change recursive algorithm can also be
  inefficient.
• In both cases, dynamic programming is the answer
  to a more efficient algorithm, using sub-problems
  toward the solution to the more complex problem.

50
Coin-change example 1
For coin denominations 1c, 5c, 10c, 25c, the
minimum number of coins to make 63c is 2x.25,
1x.10, 3x.01 --gt 6 coins The idea is to use the
biggest denominations first. In this case, it
guarantees getting the optimal solution. This
type of algorithm, finding the optimal
sub-solution at each step, is called a GREEDY
algorithm.
51
Coin-change example 2
For coin denominations 1c, 5c, 10c, 21c, 25c, the
minimum number of coins to make 63c is 3x.21--gt
3 coins The greedy algorithm would still give a
solution of 6 coins though, which is not
optimal. This was the problem with the Fibonacci
numbers.
52
Solution draft 1

• If a coin of exact value exists, choose it as
  solution.
• Otherwise, try
• denominations 1c 62 c and solve recursively
• denominations 2c 61c and solve recursively

This turns out to be too inefficient. WHY?
53
Solution draft 2

• If a coin of exact value exists, choose it as
  solution.
• Otherwise, try
• denominations 1c 62c and solve recursively
• denominations 5c 58c and solve recursively
• denominations 10c 53c and solve recursively
• denominations 25c 38c and solve recursively

This still turns out to be too inefficient. WHY?
54
Solution

• The problem lies in the fact that we re-compute
  the same thing many times.
• The solution lies in keeping the optimal
  solution, so far, of sub-problems in minCoins.
• We iterate through each denomination, kept in
  array coinsj, skipping denominations that are
  larger than the amount of money we are changing.
• Otherwise, we test whether the no.of coins used,
  coinsUsed , for the next combination lowers the
  minCoins obtained so far.
• lastCoin keeps track of the coins used for
  solution

55
public static void makeChange(int coins ,int
differentCoins, int maxChange, int coinsUsed
, int lastCoin ) coinsUsed0 0
lastCoin0 1 for (int cents 1 cents
lt maxChange cents) int minCoins
cents int newCoin 1 for(int j 0 j lt
differentCoins j) if (coinsj gt
cents) //denominations larger than amt
continue if (coinsUsedcents-coinsj 1
lt minCoins) minCoins
coinsUsedcents-coinsj 1 newCoin
coinsj coinsUsedcents minCoins
lastCoincents newCoin
56
6. DATA STRUCTURES

• Different problems require different data
  structures to be able to write the most efficient
  algorithms to solve them.
• Some examples of typical problems are matching
  parentheses, printer queues, Data Bases
  manipulations, Operating Systems structures,
  string searches, smallest largest item
  searches, compiler symbol table searches, bank
  queues, dictionary searches.

57

• Matching parentheses - stacks
• Printer queues - queues
• Data Bases manipulations - linked lists
• Operating Systems structures - trees
• String searches, smallest largest item
  searches - trees
• Compiler symbol table searches, dictionary
  searches - hash tables
• Bank queues - heaps (priority queues)

58




Parentheses stack
Printer queue
DB linked list
59
Unix tree structure
Root
Compiler hash table
Heap
60
Trees
President
Dean Engg
Dean AS
Dean Hum
Dean Bus
Acct
Fin
Phys
CS
Bio
Chem
Mech
Elect
Soc Sc
Educ
Art
Lang
Any number of branches of nodes any depth.
61
General Tree Structure
A
B
C
D
E
I
H
J
G
K
F
N
O
M
L
Q
P
Height A ( size tree) 4 Length of path E-O 2
f, l, p, q, n, o - leaves Depth of M 3
62
A tree structure

• A rooted tree has a root node, other nodes
  connected in pairs by edges.
• Each node, except for the root, is connected by a
  unique edge to a parent node, p.
• Each node, except for the leaf nodes, has one or
  more children, c.
• The length of a path from one node to another is
  the number of edges traversed.
• Most operating systems have directories and files
  arranged in a tree.

63
How to implement a tree

• The children of each node will be implemented in
  a linked list.
• Each node references its leftmost child and
  right sibling.
• The last child (a leaf) has no child.
• The rightmost sibling has no sibling.

64
Left child / Next sibling representation
A
C
B
F
E
D
65
Operations we would like to perform on trees

• Find a particular object
• Retrieve an object
• Remove an object
• Insert an object
• Go to the root, or the first child, or the first
  sibling
• Check if a node is valid, if the tree is empty,
  or empty it out

66
A tree interface
public interface Tree boolean isEmpty(
) void makeEmpty( ) public interface
TreePointer void insert( Object x) throws
Duplication boolean find( Object
x) void remove( Object x) throws
ItemNotFound void goToRoot( ) void firstChil
d( ) void nextSibling( ) boolean isValid(
) Object retrieve( )
67
A typical operating system hierarchy
Pseudo code to count files in each directory int
size( ) int totalSize sizeOfThisFile(
) if( isDirectory( ) ) for each file c in
this directory (for each child) totalSize
c.size( ) return totalSize
68
Printing the names of all the files in an
operating system tree

• Print current object
• If current object is a directory, process all
  children recursively at one level deeper into
  tree
• Use File in java.io and methods getName,
  isDirectory, getPath.
• Method list returns an array of filenames in
  directory.

69
Import java.io. public class FileSystem extends
File public FileSystem( String
name) super( name) public void
printName( int depth) for( int i 0 i lt
depth i ) System.out.print( \ t
) System.out.println( getName( ) ) public
void listAll listAll( 0)
Using class File in java.io and
methods getName, getPath
70
Private void listAll( int depth) printName(
depth) if (isDirectory( ) ) String entries
list( ) for( int i - 0 i lt
entries.length i ) FileSystem child
new FileSystem( getPath( )
separatorChar entries i ) child.listAll(
depth 1) public static void main(
String args ) FileSystem f new
FileSystem( , ) f.listAll ( )
71
Binary trees

• A binary tree is a tree in which each node, other
  than the leaves, has at most 2 children.
• The children can be named left right.
• The recursive definition is that it is either
  empty, or it has a root, a left sub-tree, a right
  sub-tree.
• Important examples are expression trees in
  language syntax, Huffman trees for compression
  algorithms.

72
Example of a binary treeThe expression tree
(a b) ( c - d) is an expression that can be
represented as a tree.


-
d
a
b
c
Each node has two children.
73
Another binary expression tree

a

d
(a (d ( b - c )))
74
File Compression

• There are 100 ASCII characters. These require 7
  bits to differentiate. The 8th bit is for parity
  check.
• Most characters do not appear in standard files.
  Characters have different frequencies.
• Compression tries to reduce the length of
  frequently used characters.
• Using a binary code for each frequently used
  character and representing them in a tree saves
  space and time.

75
Table of frequency code

A o indicates the left branch a 1 indicates the
right branch.
76
Huffman coding tree for data compression
algorithm
Path from root left link --gt 0 right link --gt 1
Symbols stored as leaves

0
1

0
0
1
1




1
0
0
1
0
0
1
a
e
i
s
t
sp
nl
a encoded as 000 e encoded as 001 i encoded as
010
s encoded as 011 t encoded as 100 sp encoded as
101
nl encoded as 110
77
A better tree
i
sp
e
a
t
s
nl
78
Huffmans Algorithm

• Merge lowest freq. Nodes into a tree

• Merge next lowest freq. node

• Continue

79
Other uses needed functions

• Binary search trees with insertion access
  routines.
• Tree implementations of priority queues (heaps)
  with access deletion of the minimum element
  (the one of highest priority).

80
A binary tree interface
Public interface BinaryTree boolean isEmpty(
) Comparable findMin throws
ItemNotFound Comparable findMax throws
ItemNotFound public Object
getElement() public void Object
setElement(Object x) element x public
void insert( Comparable x) throws
Duplication public void remove( Comparable x)
throws ItemNotFound public void removeMin( )
throws ItemNotFound public void makeEmpty(
) public void printTree( )
81
Creating nodes and binary treesand methods of
traversing trees
1. We create a binary node class providing for
references to the left and right child. 2. We
create a binary tree made up of binary nodes. 3.
We provide methods to traverse the tree. 4. We
include a method to duplicate sub-trees and merge
them to another node.
82
Traversal strategies

• We can traverse the tree from a node to its
  children. This is called PREORDER.
• We can traverse the tree from the children back
  up to the parent node. This is called POSTORDER.
• We can also process the left child, then the
  parent, then the right child. This is called
  INORDER. It is used for algebraic expressions.

83
Recall of structure of a general tree node
A B
B D C
C F
D E
E
F
G
First child / Next sibling representation For a
binary tree Root/leftChild/rightChild is a better
representation
84
// Binary node class stores a node in a
tree class BinaryNode BinaryNode( ) // a
null node this( null,null,null)
BinaryNode( Object theElement ) // a
leaf this( theElement, null, null)
BinaryNode( Object theElement, BinaryNode rt,
BinaryNode lt ) element theElement left
lt right rt static int size( BinaryNode
b) static int height( BinaryNode b) void
printPostOrder( ) Different void
printInOrder( ) traversal void
printPreOrder( ) strategies BinaryNode
duplicate( ) private Object element
BinaryNode left BinaryNode right
constructors
85
// Binary tree class public class
BinaryTree public BinaryTree( ) root
null public BinaryTree( Object rootItem
) root new BinaryNode( rootItem) public
void printPreOrder( ) if (root ! null)
root.printPreOrder( ) public void
printInOrder( ) if (root ! null)
root.printInOrder( ) public void
printPostOrder( ) if (root ! null)
root.printPostOrder( ) public boolean
isEmpty( ) return root null public
void makeEmpty( ) return root null
constructors
Traversal strategies
86
public void merge( Object rootItem, BinaryTree
t1, BinaryTree t2) public int size(
) return BinaryNode.size( root) public
int height( ) return BinaryNode.height(
root) private BinaryNode root
87
Merging sub-trees
t1
rootItem
t2
root new BinaryNode(rootItem, t1.root, t2.root)
The problem is that the nodes A - F now appear
in 2 trees. A is really t1.root, D is
t2.root. We can remedy that by setting t1.root
null, t2.root null, after we have merged them
to the rootItem.
88
public void Merge( Object rootItem, BinaryTree
t1, BinaryTree t2) if( t1.root t2.root
t1.root ! null) System.err.println(
Sub-trees are the same. Abort merge) return
root new BinaryNode( rootItem, t1.root,
t2.root) // Make sure that every node is in a
unique tree if( this ! t1) t1.root
null if( this ! t2) t2.root null
89
Computing size height of a tree
Static int size( BinaryNode) if( t
null) return 0 else return 1 size( t.left)
size( t.right) Static int height(
BinaryNode) if( t null) return
0 else return 1 Math.max( height( t.left) ,
height( t.right) ) Postorder
traversal methods.
90
Complexity of tree traversal methods

• Whether in preorder (from parent to children),
  postorder (from children to parent), or inorder
  (left child, parent, right child), each node is
  processed once, each if statement is executed
  at most once per node.
• Therefore, regardless of strategy, complexity is
  O(N).
• Iterative recursive implementations are both
  O(N).

91
Traversal Routes
1
2
3
4
6
5
7
Root - left subtree - right subtree
Left leaf - right leaf - root of subtree
Left subtree - root - right subtree
92
How to go from one Binary node to another in any
traversal
To go backwards, virtual double links are created
by stacking nodes retrieving them in the order
desired.
93
Node visitation for each traversal method

• For PreOrder traverse, treat root stack left
  subtree, treat left subtree root stack its left
  subtree stack its right subtree treat right
  subtree root
• For PostOrder traverse, stack root stack left
  subtree, then stack right subtree treat left
  leaf then right leaf, then its root
• For InOrder traverse, treat left subtree, then
  root, then right subtree.

94

Void printPreOrder( ) System.out.println(
element) if( left ! null ) left.printPreOrder(
) if( right ! null ) right.printPreOrder(
) void printPostOrder( ) if( left ! null
) left.printPostOrder( ) if( right ! null
) right.printPostOrder( ) System.out.println(
element) void printInOrder( ) if( left !
null ) left.printIntOrder( ) System.out.println(
element) if( right ! null ) right.printInOrder
( )
95
Abstract class for tree iteration in traversal
methods

• It will declare a reference to a tree and the
  current node, and it will initialize it in a
  constructor method.
• It will provide final (constant) implementations
  for 2 methods that are constant over the tree
  hierarchy isValid retrieve.
• It will declare 2 abstract methods that will be
  implemented in each type of iterator traversal
  method first advance.
• The whole thing will be part of package
  DataStructures.
• It will import Exceptions.

96
/ Tree iterator class / abstract class
TreeIterator public TreeIterator( BinaryTree
theTree ) t theTree current
null final public boolean isValid(
) return current ! null final public
Object retrieve( ) throws ItemNotFound if(
current null ) throw new ItemNotFound(
Retrieved ) return current.element abst
ract public void first( ) abstract public void
advance( ) throws ItemNotFound protected
BinaryTree t protected BinaryNode current
97
Implementation of traversal methods

• A stack will be maintained to store the current
  state, with the current node on top of the stack.
• The root will be pushed onto the stack, then the
  left sub-tree, then the right sub-tree.
• As we process each node on top of the stack, it
  will be popped from the stack.
• A counter will be maintained.
• The counter will contain 1 if we are about to
  process the nodes left sub-tree.
• The counter will contain 2 if we are about to
  process the nodes right sub-tree.
• The counter will contain 3 if we are about to
  process the node itself.
• We may be pushing popping a null (non-existent)
  sub-tree.

98
5
PostOrder (each node traversed 3 times)
2
4

• Push root 5. Visit 5, but counter 1.
• Push back root 5 5s left sub-tree 2.
• Visit 2, but ctr 1. Push 2 2s left sub-tree
  1.
• Try 1s left sub-tree null. Visit 1 Push back.
• Try 1s right sub-tree null. Visit 1 Push back.
• Pop treat node 1.
• Try 2s right sub-tree null. Pop treat node 2.
• Push back root 5 5s right sub-tree 4.
• Visit 4, but ctr 1. Push 4 4s left sub-tree
  3.
• Try 3s left sub-tree null. Visit 3 Push back.
• Try 3s right sub-tree null. Visit 3 Push back.
• Pop treat node 3.
• Try 4s right sub-tree null. Pop treat node 4.
• Counter for 5 3. Pop treat node 5.

1
3
99
/ PostOrder iterator class / public class
PostOrder extends TreeIterator protected
static class StNode BinaryNode node int
timesPopped StNode( BinaryNode n) node n
timesPopped 0 public PostOrder(
BinaryTree theTree) super( theTree) s
new StackAr( ) s.push( new StNode( t.root)
) public void first( ) s.makeEmpty(
) if( t.root ! null) s.push( new StNode(
t.root) ) try advance( ) catch(
ItemNotFound e) ( ) protected Stack s
100
/ Advance current position to next node
/ public void advance( ) throws
ItemNotFound if( s.isEmpty( ) ) if ( current
null ) throw new ItemNotFound( No more
found) current null return StNode
cnode for( ) cnode (StNode)
s.topAndPop( ) if ( cnode.timesPopped
3) current cnode.node return s.
push( cnode) if ( cnode.timesPopped
1) if ( cnode.node.left ! null ) s.push
( new StNode ( cnode.node.left ) ) else
// cnode.timesPopped 2 if (
cnode.node.right ! null ) s.push ( new
StNode ( cnode.node.right ) )
101

• InOrder (each node traversed twice)
• The difference between the PostOrder InOrder
  traversal is that in the InOrder
• a node is declared visited after it has been
  popped twice.
• The right child is then pushed onto the stack,
  so that the advance method will treat it next.

3
2
5
1
4
102
public class InOrder extends PostOrder
public class InOrder ( BinaryTree theTree )
super ( theTree) public void advance(
) throws ItemNotFound if( s.isEmpty( )
) if ( current null ) throw new
ItemNotFound( No more items found) current
null return StNode cnode for(
) cnode s(StNode).topAndPop( ) if (
cnode.timesPopped 2) current
cnode.node if ( cnode.node.right ! null
) s.push ( new StNode ( cnode.node.right )
) return s.push( cnode) if (
cnode.node.left ! null ) s.push ( new StNode
( cnode.node.left ) )
103
PreOrder (each node is popped when first visited)
1
2
4
3
5

• The PreOrder traversal differs from the InOrder
  in that
• a node is declared visited after it is popped
  once
• it then pushes the right child onto the stack,
  then the left child

104
public class PreOrder extends TreeIterator publ
ic PreOrder ( BinaryTree theTree ) super (
theTree ) s new StackAr( ) s.push (
t.root ) public void first ( )
s.makeEmpty ( ) if ( t.root ! null ) s.push
( t.root) try advance ( ) catch (
itemNotFound e ) ( ) public void advance ( )
throws ItemNotFound private Stack s
105
public void advance( ) throws ItemNotFound if(
s.isEmpty( ) ) if ( current null
) throw new ItemNotFound( No more items
found) current null return current
( BinaryNode ) s.topAndPop ( ) if (
current.right ! null ) s.push ( current.right
) if ( current.left ! null ) s.push (
current.left )
106
Breadth-first search or level order traversal
We can traverse the tree level by level. This is
used mostly in breadth-first searches in AI -
Artificial Intelligence.
1
This is implemented as a queue.
107
public class levelOrder extends TreeIterator
public levelOrder ( BinaryTree theTree )
super ( theTree ) q new QueueAr (
) q.enqueue ( t.root ) public
void first ( ) q.makeEmpty ( ) if
( t.root ! null ) q.enqueue ( t.root )
public void advance ( ) throws
ItemNotFound private queue q
108
Public void advance ( ) throws ItemNotFound
if ( q.isEmpty ( ) ) if ( current
null) throw new ItemNotFound ( No more
items ) current null return
current q( BinaryNode ).dequeue (
) if ( current.left ! null
) q.enqueue ( current.left ) if (
current.right ! null ) q.enqueue (
current.right )
109
Binary Search Tree

• What use is it?
• To implement binary searches
• Why?
• It is of logarithmic complexity
• Who cares? Or why not an O(N) algorithm?
• If you count votes in NYS you need 20x106 record
  accesses to check for eligibility
• So what?
• You can do 25x106 instructions or 120 disc
  accesses in 1 second.

110
Examples of uses of Binary trees

• For a client DB we might want to look up max.
  sale in month
• For supplier DB we might want to look up min.
  cost item
• We want a tree that permits looking for mid
  quickly, then restricting search to half the
  tree, then looking for mid again in whats left
  of the tree, ...

111
Technique
2
x gt mid
3
x lt mid
1
We want min. values on left sub-tree, max. values
on right sub-tree
112
Example Find 12
32
16
48
8
64
24
40
4
12
findMin and findMax of complexity O(log N)
113
In general
a
blta
ggta
b
g
cltb
fgtb
hltg
igtg
c
i
f
h
dltc
egtc
d
e
InOrder traversal
114
INSERT

• If we get a new sale amount for 28, where do we
  insert it?

• Go down tree, left if smaller, right if bigger.

• Method addNode takes time t where
• O(log N) lt t lt O(N)
• depending on shape of the tree.

115
O(N) or O(log N)
Complexity O(logN)
Complexity O(N)
116
REMOVE
If we remove a sale amount, say 48, what happens
to nodes 40 64? How about removing node 40?

• If leaf is removed, no problem.
• If root 4 of sub-tree with one child 3,
• remove 4, attach 3 to 2.
• If root of sub-tree with 2 children?

117
If root of sub-tree with 2 children?
Replace removed node with smallest item in right
sub-tree. This necessitates knowing the size of
sub-trees.
118
Find, findMin, findMax
119
Find method
protected BinaryNode find ( Comparable x,
BinaryNode t) throws ItemNotFound while ( t !
null ) if ( x.compares ( t.element ) lt 0 ) t
t.left else if ( x.compares ( t.element ) gt
0 ) t t.right else return t throw
new ItemNotFound ( None found)
120
findMin Method
protected void findMin ( BinaryNode t ) throws
ItemNotFound if ( t null) throw new
ItemNotFound (None found while ( t.left !
null ) t t.left return t
121
findMax Method
protected void findMax ( BinaryNode t ) throws
ItemNotFound if ( t null) throw new
ItemNotFound (None found while ( t.right !
null ) t t.right return t
122
Insert
123
Insert Method
protected void insert ( Comparable x, BinaryNode
t ) throws DuplicateItem if ( t null ) t
new BinaryNode ( x, null, null ) else if (
x.compares ( t.element ) lt 0 ) t.left insert
( x, t.left ) else if ( x.compares ( t.element
) gt 0 ) t.right insert ( x, t.right
) else throw new DuplicateItem ( Duplicate
item found)
124
Remove a one child root
Make child of removed node right child of removed
nodes parent, if it was a right child. Make
child of removed node left child of removed
nodes parent, if it was a left child.
125
removeMin method - for a leaf
protected void removeMin ( BinaryNode t ) throws
ItemNotFound if ( t null ) throw new
ItemNotFound ( No item found) if ( t.left !
null) t.left removeMin ( t.left ) else t
t.right return t
126
Remove a two children root
Replace removed root (red node) by left child of
its right child Or right child of its left child.

127
remove method - with 2 children
Protected BinaryNode remove ( Comparable x,
BinaryNode t ) throws ItemNotFound if ( t
null ) throw new ItemNotFound( None found
) if ( x.compares ( t.element ) lt 0 ) t.left
remove ( x, t.left ) else if ( x.compares (
t.element ) gt 0 ) t.right remove ( x, t.right
) else if ( t.left ! null t.right !
null) t.element findMin ( t.right
).element t.right removeMin ( t.right
) else t ( t.left ! null ) ? t.left
t.right return t
128
Example of removing kth min item

• Suppose we have a DB with keys to each days
  orders from customers, and the remaining number
  of each item sold.
• Suppose you have to do an inventory update on the
  fifth day of the month, each month, in order to
  order more merchandize from suppliers.
• You might want to find the least five numbers of
  items to buy this month.
• You will need to be able to find the first,
  second, third, fourth and fifth least number.

129
How to find Kth min. item

• There are 3 possible cases
• K SL1 so K is the root of the tree.
• K lt SL then K is in left sub-tree can be
  found recursively.
• Otherwise K is the (K SL 1)th least item in
  the right sub-tree.

Size of left subtree is SL 4
The method is to keep the node size for each
insert and each remove. So we will override the
insert, remove, removeMin methods.
130
public class rankedBinarySearchTree extends
BinarySearchTree public Comparable findKth
(int k) throws ItemNotFound return findKth (
k, root).element protected
BinaryNode findKth (int k, BinaryNode t) throws
ItemNotFound if ( t null ) throw new
ItemNotFound (Not found) int leftSize (
t.left !null) ? t.left.size 0 if ( k lt
leftSize) return findKth ( k, t.left ) else
if ( k leftSize 1 ) return t else
return findKth ( k leftSize 1, t.right
)
131
Redefined Binary Nodes
Class BinaryNode BinaryNode ( Comparable e
) this( e, null, null) BinaryNode (
Comparable e, BinaryNode lt, BInaryNode
rt) element e left lt right
rt Comparable element BinaryNode left
BinaryNode right int size 1
132
Overridden insert method
protected BinaryNode Insert (Comparable x,
BinaryNode t ) throws DuplicateItem if ( t
null ) return new BinaryNode ( x, null, null
) else if ( x.compares ( t.element ) lt 0
) t.left insert ( x, t.left) else if (
x.compares ( t.element ) gt 0 ) t.right insert
( x, t.right) else throw new DuplicateItem
(Ranked insert) t.size return t
133
Overridden removeMin method
protected BinaryNode removeMin(BinaryNode t )
throws ItemNotFound if ( t null ) throw
new ItemNotFound( Ranked removeMin) if (
t.left null ) return t.right t.left
removeMin ( t.left) t.size - - return
t
134
protected BinaryNode remove (Comparable x,
BinaryNode t ) throws ItemNotFound if ( t
null ) throw new ItemNotFound ( Not
Found) if ( x.compares ( t.element ) lt 0
) t.left remove ( x, t.left) else if (
x.compares ( t.element ) gt 0 ) t.right remove
( x, t.right) else if ( t.left ! null
t.right ! null ) t.element findMin (
t.right ).element t.right removeMin (
t.right ) else return ( t.left ! null ) ?
T.left t.right t.size -- return t
135
Complexity of Binary Search Tree operations

• The cost of each operation is proportional to the
  number of accesses.
• So each nodes cost is (1 its depth).
• The depth depends on whether each node has 2
  children (called a totally balanced tree) or each
  node has 1 child (called a totally unbalanced
  tree).
• The depth of a balanced tree is (log N), that of
  an unbalanced tree is (N-1).

136
AVL Trees

• The Adelson-Velskii-Landis trees are balanced but
  not totally. That makes insertion removal
  easier.
• They still permit searches in logarithmic time.
• For any node in an AVL Tree, the height of the
  left and right sub-trees can differ by at most 1.
• Because the height, H, of a AVL tree has at least
  FH3 -1 nodes, where Fi is the ith Fibonacci
  number, it can be proven that a search of an AVL
  tree is of logarithmic complexity.

137
Example
Non-AVL unbalanced tree
AVL tree
Insertion or deletion may destroy balance !!!
138
Problems encountered in insertions deletions
A
Insertion in right sub-tree of left child of X
Insertion in left sub-tree of left child of X
Insertion in right sub-tree of right child of X
Insertion in left sub-tree of right child of X
As height would be 2 more than its sibling, and
the tree is no longer an AVL tree.
139
Solutions
Solution depends on the fact that, as long as we
keep the relationship between the values in the
nodes, we can interchange the branches of a
Binary Search Tree
If b replaces a, a replaces g, relative values
remain the same !!!
140
Solution by single rotation
For the left sub-tree of the left child or the
right sub-tree of the right child a single
rotation of the tree will not change the relative
values of the nodes.
Left tree is not an AVL tree. Right tree is now
an AVL tree .
Left child of root becomes root right child of
left child becomes left child of right child
141
Single rotation for ltlt child
Static BinaryNode LtLtChild BinaryNode a
) BinaryNode b a.left a.left
b.right return b
142
Example after insertion of node 1
Unbalanced node
143
Single rotation for rt-rt child
Static BinaryNode RtRtChild BinaryNode a
) BinaryNode c a.right a. right c.
left c.left a return c
144
Example of problems with a single rotation of a
lt-rt subtree
145
Example of double rotation for lt-rt subtree
part 1
Left rotation of right subtree
146
Example of double rotation for lt-rt subtree
part 2
Right rotation of left subtree
147
Problems with single rotations for lt-rt and
rt-lt children
Height of left and right sub-trees still differ
by gt1
148
Double rotation for rt-lt lt-rt children
We rotate nodes such that the right child follows
the right rotation (because it is greater), but
the left child remains behind (because it is
smaller).
149
Example of double rotationphase 1
8.left RtRtChild ( 8.left)
150
Example of double rotationphase 2
return LtLtChild ( 6 )
151
Code for insertion into right sub-tree of left
child
static BinaryNode doubleLtRt (BinaryNode
k4) k4.left RtRtChild ( a.left) return
LtLtChild ( k4 )
152
Code for insertion into left sub-tree of right
child
static BinaryNode doubleRtLt (BinaryNode
k3) k3.right LtLtChild ( b.right) return
RtRtChild ( k3 )
153
Binary Heap

• A priority queue allows access to the min (or
  max) item
• The Binary Heap is a priority queue that allows
  insertion of new items the deletion of the
  minimum item in logarithmic time.
• The Binary Heap is implemented by an array.
• To implement a priority queue we can use a
  linked-list with insertions on front done in
  constant time, and deletions in linear time, or
  vice-versa.
• We implement a Binary Heap with a simple array,
  which supports insert and deleteMin in O(log N)
  in the worst case it supports insert in O(c) in
  average case it supports findMin in O(c) in
  worst case.

154
Balanced Tree structure of the Binary Heap
In order to get log complexity in the worst case,
the Binary Heap is organized as a balanced tree,
while it is implemented as an array.
155

• The correspondence between an array and a
  balanced tree works if
• the tree is a complete Binary Tree - all levels
  except the leaves must be filled
• there cannot be gaps in the nodes, including the
  leaves - J in the example must be a left child

• The worst case log complexity works because
• for N nodes and height H we must have
• 1 2H-1 ? N ? 1 2 4 2H
• 1 2H-1 ? N ? 2H1 - 1
• Therefore the height is at most log N

156

• In a complete Binary Tree we do not need a left
  child right child reference.
• This allows us to store each level in
  consecutive positions in an array.
• If we start the root node at position 1 rather
  than 0, then for every node in position i will
  have its left child in position 2i and its right
  child in position 2i1.
• If either position 2i or 2i1 extends past the
  number of nodes, we know that that child does not
  exist.
• Inversely, given a node in position i, we know
  that its parent node will be in position i/2.
• The root node will have a virtual parent in
  position 1/2 0. Another reason to start the
  root at position 1.
• In order to retrieve min or max, each parent
  node will be smaller than the children nodes.

157

• For many inserts it is cheaper to just toss an
  item and then reinstate order with fixHeap. This
  is done in linear time.
• To keep track of that, orderOK is set to false
  when a toss is performed.

158
Public class BinaryHeap implements
PriorityQueue public BinaryHeap(
) public void insert( Comparable x) //
inserts in right order public void toss
(Comparable x) // inserts in any
order public Comparable findMin( )
throws Underflow public Comparable
deleteMin( ) throws Underflow public
boolean isEmpty ( ) return currentSize 0
public void makeEmpty ( ) currentSize 0
public int size ( ) return currentSize
private void getArray ( int newMaxSize
)
159
// To insert a node in tree we find first hole
percolate up to keep order private void
buildHeap( ) private void percolateDown
( int hole ) private void fixHeap ( )
//reinstates heap order after toss
private int currentSize private Comparable
array // if heap has (parent node gt
children nodes) after a toss private boolean
orderOK // set to false when toss is
performed private static final int
DEFAULT_CAPACITY
160
public BinaryHeap ( ) currentSize 0 array
new Comparable DEFAULT_CAPACITY 1 /
Construct Binary Heap from array / public
BinaryHeap ( Comparable items
) currentSize items.length array new
Comparable items.length 1 for ( int i
0 i lt items.length i 1) array i 1
items i buildHeap ( )
161
Public BinaryHeap ( Comparable negInf
) currentSize 0 orderOK true getArray
(DEFAULT_CAPACITY) array0 negInf Public
Comparable findMin ( ) throws Underflow if (
isEmpty ( ) ) throw new Underflow( Empty
binary heap ) if ( !orderOK) fixHeap (
) return array1 Private void getArray (
int newMaxSize) array new Comparable
newMaxSize 1
162
Insert

• Inserting an item into the heap implies adding a
  node to the tree.
• To preserve the completeness of the tree, we
  position ourselves at the next available
  location.
• If inserting there does not respect the
  parent-child order, we slide its parent to that
  position (hole).
• If parent is not in the right order, we slide
  ITS parent, so on.
• Eventually all parents have slid into the right
  positions, leaving a hole.
• The node to be inserted takes that position,
  called the hole.
• This technique is called percolating up.
• This is done in logarithmic time.

163
View of Inserting 14
31
164
Private void checkSize ( ) if (currentSize
array.size - 1) Comparable oldarray
array getArray( currentSize 2) for (int
i0 i lt oldArray.length i1) arrayi
oldArrayi public void toss (Comparable
x) checkSize( ) arraycurrentSize
x if (x.compareTo (arraycurrentSize/2) lt 0
) orderOK false
165
Public void insert (Comparable x) if
(!orderOK) toss (x) return CheckSize
( ) // percolate up int hole
currentSize for ( x..compareTo(arrayhole/2
) lt 0 hole / 2 ) array hole array
hole/2 array hole x
166
deleteMin

• deleteMin creates the reverse problem from
  insert it creates a hole and destroys the
  completeness of the tree.
• Since the min is the root of the tree, we have
  to percolate down the tree, moving the hole to
  the last leaf on the tree.
• This requires two methods, one to remove the
  smallest item at the root, the other to percolate
  the hole down to the last leaf.
• This is done in logarithmic time.

167
View of deleteMin
168
(No Transcript)
169
public Comparable deleteMin ( ) throws
Underflow Comparable minItem findMin (
) array 1 array currentSize
-- percolateDown (1) return
minItem private void percolateDown (int
hole) int child Comparable temp array
hole for ( hole 2 lt currentSize hole
child) child hole 2 if (child
! currentSize array child1.compareTo
(array child ) )lt0 child if (array
child.compareTo (temp) lt 0 ) array hole
array child else break array hole
temp
170
FixHeap

• An insertion can be done in O(log N)
• N insertions can be done in O(N logN)
• When N insertions must be done, it is better to
  toss the items into the heap and then fix the
  heap.
• We could exchange higher parents with smaller
  children, going recursively down left right,
  using percolateDown.
• It is cheaper to exchange from the lowest
  parents up.So we use reverse level order
  traversal.

171
View of fixHeapwith reverse level traversal

• Check lowest parent on right, 68, its
  children, 81 73. Order is OK.
• Check next lowest parent, 49, its children, 61
  30.
• Exchange nodes 49 30 percolateDown.

• Check next lowest parent, 21, its children, 32
  31. Order is OK.
• Check next lowest parent, 19, its children, 60
  26. Order is OK.
• Check next lowest parent, 16, its children, 30
  68. Order is OK.
• Check next lowest parent, 45, its children, 19
  21. Exchange nodes 45 19 then percolateDown.
  Check 45 with 60 26. Exchange 45 26.
• Check next lowest parent, 88, its children, 21
  16. percolateDown exchange nodes 88 21 then
  percolateDown.Exchange 88 21. percolateDown
  then exchange 88 45.

172
Final tree
173
FixHeap Algorithm
Private void fixHeap ( ) for ( int i
currentSize/2 i gt 0 i --) percolateDown
(i) orderOK true
174
BUILD HEAP
Private void buildHeap ( ) for ( int i
currentSize / 2 i gt 0 i-- ) percolateDown ( i
)
Heap Sort
Toss each item in Binary heap Apply buildHeap
with max on top in an array starting at 0. Call
deleteMin N times, using space located to store
ordered items
Public static void heapsort (Comparable a
) for ( int I a.length /2 i gt 0 i --
) percDown ( a, i, a.length ) for ) int j
a.length 1 j gt 0 j-- ) swapReferences ( a,
0, j ) precDown ( a, 0, j )
175
SORTING

• There different types of sorting algorithms of
  different complexities.
• For a short array of items, the simplest
  algorithm will do, even if it is of worst
  complexity.
• For a partially sorted array, or a sorted array
  to which we add one element, the simplest
  algorithm will work at low complexity.

176
Types of sorts

• Insertion sort (complexity ?(N) or O(N2) )
• Shell sort (complexity O(N3/2) O(N2) )
• Merge sort ( complexity O(N log N) )
• Quick sort ( complexity O(N log N) ) but the
  running time is faster than that of Merge sort.

177
Insertion Sort
178
Code evolution
temp ap
If ap lt ap-1 ? ap ap-1
If ap lt ap-1 ? ap ap-1 ap-1
temp
ap temp
179
Code evolution
Public static void insertionSort (Comparable a
) for ( int p 1 p lt a.length p )
Comparable temp ap int j p for ( j gt 0
temp.lessThan ( aj-1 ) j-- ) aj
aj-1 aj temp

180
Complexity

• If array is not sorted, the nested loops are
  executed N x N times. Therefore, for unsorted
  arrays the complexity is O(N2).
• For a sorted array into which we insert a new
  item, at worst one must compare each item with
  the one being inserted, and then the insertion
  takes place. Therefore the complexity is O(N).

181
Shell sort

• The general idea of the Shell sort is that we
  look at the first element and the kth element,
  and sort them.
• Then we look at the 2nd element and the (k1)th
  element, and sort them.
• Then we look at the 3rd element and the (k2)th
  element and sort them.
• .
• At the end we start over again with a gap lt k.
• We end when the gap is 1, which is the Insertion
  Sort applied to an array which is almost
  completely sorted.
• It can be shown that the complexity of the Shell
  sort is
• O(N3/2) lt complexity lt O(N2)
• It can be shown that the best start is a gap of
  N/2, and subsequent gaps can be N/2.2

182
Shell sort after a first pass with a gap of 5
81 94 11 96 12 35 17
95 28 58 41 75 15
183
Shell sort after a first pass with a gap of 3
35 17 11 28 12 41 75
15 96 58 81 94 95
184
Shell sort after a first pass with a gap of 1
28 12 11 35 15 41 58
17 94 75 81 96 95
...
11 12 15 17 28 35 41
58 75 81 94 95 96
Remember the best first gap is N/2, and the
best subsequent gaps areN/2.2
185
public static void shellSort (Comparable a
) for ( int gap a.length/2 gap gt 0 gap
gap 2 ? 1 (gap / 2.2) ) for ( int
i gap i lt a.length i )
Comparable temp a i int j i for
( j gt gap temp.lessThan (a j - gap) j -
gap ) a j a j - gap a j
temp
186
Complexity of Shell sort

• If we take N/2 as the first gap and (N/2) 1 for
  subsequent gaps, the worst case is the O(N3/2).
  The average case is around O(N5/4).
• If we take N/2 as the first gap and (N/2.2) for
  subsequent gaps, the average case is then below
  O(N5/4).

187
Merge Sort

• You take an array of N numbers. You divide it
  into 2 arrays of N/2 numbers.
• You recursively halve the arrays until you are
  left with arrays of 2 numbers each.
• You sort the 2 numbers arrays.
• You merge the 2 numbers arrays, then the 4
  numbers arrays, etc.

188
Example of partitioning
189
Merging example
190
Complexity of merge sort

• The halving of the arrays sorting them is done
  in O(log N).
• The merging is done in O(N).
• So the Merge Sort is done in O(N log N).

191
private static void mergeSort ( Comparable a ,
Comparable tmpAr