CSC 332 Algorithms and Data Structures

1
CSC 332 Algorithms and Data Structures

• Binary Search Trees
• Unbalanced and Balanced

Dr. Paige H. Meeker Computer Science Presbyterian
College, Clinton, SC (Portions of these notes
from Weiss book and University of Washington
Website)
2
Binary Search Trees

• Goal Create a simple data structure that will
  allow insertions and deletions to be O(log N) on
  average.
• (Unfortunately, without tweaking, these
  operations also take O(N) worst case, though they
  are O(log N) on average.

3
Binary Search Tree (BST)

• For any node in the BST, all smaller nodes are in
  the left subtree and all larger nodes are in the
  right subtree. Duplicates are not allowed.

4
BST Operations

• insert
• find
• findMin
• findMax
• remove
• removeMin
• removeMax
• How would we implement these?

5
BST Operations

• find
• Start at the root and repeatedly branch to either
  the left or right, depending on a comparison with
  the element in the node and the element we are
  searching for.
• Can now either return true (found), false (not
  found), or insert the value at this point.

6
BST Operations

• find - insert
• To insert a node, use the find method and if
  null is found, insert at that point.

7
Example Growing a Binary Search Tree
7
1. insert 7
3
9
2. insert 3
3. insert 9
5
4. insert 5
5. insert 6
6
8
BST Operations

• findMin
• Start at the root and repeatedly branch left
  until we run out of left children. The stopping
  point is the smallest element

9
BST Operations

• findMax
• Start at the root and repeatedly branch right
  until we run out of right children. The stopping
  point is the largest element

10
BST Operations

• remove
• Most difficult operation
• Non-leaf nodes hold the tree together and we
  dont want to disconnect the tree
• We need to keep the tree connected and maintain
  the BST property, while avoiding making the tree
  unnecessarily deep.

11
BST Operations

• remove
• Consider
• Node is a leaf
• Node has one child
• Node has two children

12
BST Operations

• remove
• Node is a leaf easy! Just remove it. Its
  removal does not disconnect the tree.

13
BST Operations

• remove
• Node has one child not too difficult.
• Node can be removed after adjusting its parents
  child link to bypass the node.

14
BST Operations

• remove
• Node has two children most difficult case
• Node is replaced by using the smallest item in
  the right subtree and then removing that item
• The smallest item in the right subtree is either
  a leaf (simple remove) or has a right subtree of
  its own, (a one child removal)
• (See Example)

15
BST Remove Node with 2 Children
16
BST Remove Node with 2 Children
17
BST Implementation

• In order to implement a BST, we need to
  understand the concept of a node of the tree
  a data structure that will contain references to
  another object as well as to two additional
  nodes. Your author created a small, simple class
  to serve this purpose.
• The same purpose could be served if the
  BinaryNode class was an inner class.

18
Class BinaryNode

• class BinaryNodeltAnyTypegt
• BinaryNode( AnyType theElement )
• element theElement
• left right null
• // Data accessible by other package routines
• AnyType element // The data in
  the node
• BinaryNodeltAnyTypegt left // Left child
• BinaryNodeltAnyTypegt right // Right child

19
Class BinarySearchTree

• // CONSTRUCTION with no initializer
• //
• // PUBLIC OPERATIONS
• // void insert( x ) --gt Insert x
• // void remove( x ) --gt Remove x
• // void removeMin( ) --gt Remove minimum item
• // Comparable find( x ) --gt Return item that
  matches x
• // Comparable findMin( ) --gt Return smallest
  item
• // Comparable findMax( ) --gt Return largest item
• // boolean isEmpty( ) --gt Return true if
  empty else false
• // void makeEmpty( ) --gt Remove all items
• // ERRORS
  
• // Exceptions are thrown by insert, remove, and
  removeMin if warranted
• /
• Implements an unbalanced binary search tree.
• Note that all "matching" is based on the
  compareTo method.
• _at_author Mark Allen Weiss
• /

20
Class BinarySearchTree

• Class contains both public and private methods.
  The public methods have implementations to call
  the hidden ones. The only data member is the
  reference to the root of the tree root. If the
  tree is empty, the root is null.

21
Class BinarySearchTree Public Methods

• public BinarySearchTree( ) // Construct the
  tree
• root null
• public void insert( AnyType x ) // insert x
  into the tree
• root insert( x, root )
• public void remove( AnyType x ) // remove x
  from the tree
• root remove( x, root )
• public void removeMin( ) // remove minimum item
  from the tree
• root removeMin( root )
• public AnyType findMin( ) // find smallest
  item in the tree
• return elementAt( findMin( root ) )
• public AnyType findMax( ) // find the largest
  item in the tree
• return elementAt( findMax( root ) )
• public AnyType find( AnyType x ) // find x in
  the tree

22
Class BinarySearchTree

• Next, there are several methods that operate on a
  node passed as a parameter. The idea is that the
  publicly visible routines call these hidden
  routines and pass root as a parameter. These
  hidden routines do all the work.

23
Class BinarySearchTree Private and Protected
Methods

• /
• Internal method to get element field.
• _at_param t the node.
• _at_return the element field or null if t is
  null.
• /
• private AnyType elementAt( BinaryNodeltAnyTypegt
  t )
• return t null ? null t.element

24
Class BinarySearchTree Private and Protected
Methods

• /
• Internal method to insert into a subtree.
• _at_param x the item to insert.
• _at_param t the node that roots the tree.
• _at_return the new root.
• _at_throws DuplicateItemException if x is
  already present.
• /
• protected BinaryNodeltAnyTypegt insert( AnyType
  x, BinaryNodeltAnyTypegt t )
• if( t null )
• t new BinaryNodeltAnyTypegt( x )
• else if( x.compareTo( t.element ) lt 0 )
• t.left insert( x, t.left )
• else if( x.compareTo( t.element ) gt 0 )
• t.right insert( x, t.right )
• else
• throw new DuplicateItemException(
  x.toString( ) ) // Duplicate
• return t

25
Class BinarySearchTree Private and Protected
Methods

• /
• Internal method to remove from a subtree.
• _at_param x the item to remove.
• _at_param t the node that roots the tree.
• _at_return the new root.
• _at_throws ItemNotFoundException if x is not
  found.
• /
• protected BinaryNodeltAnyTypegt remove( AnyType
  x, BinaryNodeltAnyTypegt t )
• if( t null )
• throw new ItemNotFoundException(
  x.toString( ) )
• if( x.compareTo( t.element ) lt 0 )
• t.left remove( x, t.left )
• else if( x.compareTo( t.element ) gt 0 )
• t.right remove( x, t.right )
• else if( t.left ! null t.right !
  null ) // Two children
• t.element findMin( t.right
  ).element
• t.right removeMin( t.right )

26
Class BinarySearchTree Private and Protected
Methods

• /
• Internal method to remove minimum item
  from a subtree.
• _at_param t the node that roots the tree.
• _at_return the new root.
• _at_throws ItemNotFoundException if t is
  empty.
• /
• protected BinaryNodeltAnyTypegt removeMin(
  BinaryNodeltAnyTypegt t )
• if( t null )
• throw new ItemNotFoundException( )
• else if( t.left ! null )
• t.left removeMin( t.left )
• return t
• else
• return t.right

27
Class BinarySearchTree Private and Protected
Methods

• /
• Internal method to find the smallest item
  in a subtree.
• _at_param t the node that roots the tree.
• _at_return node containing the smallest item.
• /
• protected BinaryNodeltAnyTypegt findMin(
  BinaryNodeltAnyTypegt t )
• if( t ! null )
• while( t.left ! null )
• t t.left
• return t

28
Class BinarySearchTree Private and Protected
Methods

• /
• Internal method to find the largest item
  in a subtree.
• _at_param t the node that roots the tree.
• _at_return node containing the largest item.
• /
• private BinaryNodeltAnyTypegt findMax(
  BinaryNodeltAnyTypegt t )
• if( t ! null )
• while( t.right ! null )
• t t.right
• return t

29
Class BinarySearchTree Private and Protected
Methods

• /
• Internal method to find an item in a
  subtree.
• _at_param x is item to search for.
• _at_param t the node that roots the tree.
• _at_return node containing the matched item.
• /
• private BinaryNodeltAnyTypegt find( AnyType x,
  BinaryNodeltAnyTypegt t )
• while( t ! null )
• if( x.compareTo( t.element ) lt 0 )
• t t.left
• else if( x.compareTo( t.element ) gt 0
  )
• t t.right
• else
• return t // Match
• return null // Not found

30
BST Analysis of Operations

• Cost of each BST operation is proportional to the
  number of nodes accessed during the operation.
• Cost of access of any node in the BST is 1depth
  of the node.
• This cost is logarithmic for a well balanced
  tree, but can be as bad as linear. (How? What
  would the tree look like?)

31
BST Analysis of Operations

• The internal path length of a binary tree is the
  sum of the depths of its nodes
• The internal path length is used to measure the
  cost of a successful search.

32
BST Analysis of Operations

• The external path length of a binary tree is the
  sum of the depths of the N1 null links. The
  terminating null node is considered a node for
  these purposes.
• The external path length is used to measure the
  cost of an unsuccessful search.

33
BST Analysis of Operations

• What type of input would cause our worst case
  tree?

34
BST Analysis of Operations

• How do we solve the problem?
• We must insist on balance no node is allowed to
  get too deep.
• Several algorithms can be used to implement a
  balanced binary search tree, which guarantees
  logarithmic depth in the worst case.
• Plus? Faster access time, Protection against
  poor performance, Searching averages 25 faster.
• Minus? More complicated to implement, longer
  insertions and deletions

35
Balanced BSTs

• AVL Trees
• Red-Black Trees
• AA Trees

36
AVL Trees

• First balanced BST
• Named after its discoverers, Adelson-Velskii and
  Landis
• Balance condition ensures that depth of the tree
  is O(logN).

37
AVL Trees

• An AVL Tree is a binary search tree with the
  additional balance property that, for any node in
  the tree, the height of the left and right
  subtrees can differ by at most 1.
• Updates in AVL trees could destroy the balance
  therefore, before the operation is complete, the
  tree must be rebalanced if necessary.

38
AVL Trees

• So, we must modify insert and remove to rebalance
  the tree if necessary
• We must also keep track of the height balance
  factor of the subtrees.
• Only nodes on the path from the root to the
  insertion point can have their balances altered,
  because only those nodes have their subtrees
  altered.

39
AVL Trees Balance Factor

• balance factor, for a tree node n
• height of n's right subtree minus height of n's
  left subtree
• BFn Heightn.right - Heightn.left
• a binary search tree is an AVL tree if
• balance factor of each node is 0, 1 or -1(if no
  node's two child subtrees differ in height by
  more than 1)

40
AVL Rebalancing

• As we follow the path up to the root from the
  insertion/deletion of a node, and update the
  balances, we may find a node that violates the
  AVL condition. Rebalancing the tree at this
  first node (i.e. deepest) will guarantee that the
  tree is an AVL tree once more.
• Do we believe this?? (We should!)

41
AVL Rebalancing

• There are four cases to consider that causes the
  AVL tree to become unbalanced at node X
• An insertion into the left subtree of the left
  child of X
• An insertion into the right subtree of the left
  child of X
• An insertion in the left subtree of the right
  child of X
• An insertion in the right subtree of the right
  child of X
• Cases 1 and 4 are mirror images, as are 2 and 3.

42
AVL Rebalancing

• Balance is restored by tree rotations. There are
  two types
• Single rotation switches the roles of the
  parent and child while maintaining the search
  order.
• Double rotation involves 3 nodes and 4
  subtrees equivalent to two single rotations.

43
AVL Rebalancing

• Single rotations occur when the insertion occurs
  on the outside (left,left or right,right)
• Double rotations occur when the insertion occurs
  on the inside (left,right or right,left) and is
  more complex.

44
AVL Tree Examples

• Two binary search trees
• (a) an AVL tree
• (b) not an AVL tree (unbalanced nodes are
  darkened)

45
AVL Tree Balance Factors
1
0
-1
0
0
0
-1
0
-1
1
-1
0
0
0
0
46
Which are AVL trees?
47
Unbalanced AVL Trees How to fix???
-1
-2
-2
-1
2
-1
2
0
0
1
0
-1
0
0
48
Problem cases for AVL add
1. insertion into left subtree of node's left
child 2. insertion into right subtree of node's
left child ...
49
Insertion problem cases, contd
3. insertion into left subtree of node's right
child 4. insertion into right subtree of node's
right child
50
AVL tree data structure

• potential balancing problems occur when a new
  element is added or removed
• Maintain balance using rotations
• the idea reorganize the nodes of an unbalanced
  subtree until they are balanced, by "rotating" a
  trio of parent - leftChild - rightChild
• tree will maintain its balance so that searches
  (contains) will take O(log n)

51
Right rotation to fix Case 1

• right rotation (clockwise) left child becomes
  parent original parent demoted to right makes
  original left childs right subtree the left
  subtree of the original parent.

52
Right rotation, steps

• detach left child (7)'s right subtree
  (10) (don't lose it!)
• consider left child (7) be the new parent
• attach old parent (13) onto right of new parent
  (7)
• attach old left child (7)'s old right subtree
  (10) as left subtree of new right child (13)

53
Right rotation example
54
Right rotation example
55
Left rotation to fix Case 4

• left rotation (counter-clockwise) right child
  becomes parent original parent demoted to left
  original right childs left subtree becomes the
  original parents right subtree.

56
Left rotation, steps

• detach right child (70)'s left subtree
  (60) (don't lose it!)
• consider right child (70) be the new parent
• attach old parent (50) onto left of new parent
  (70)
• attach old right child (70)'s old left subtree
  (60) as right subtree of new left child (50)

57
Problem Cases 2, 3

• a single right rotation does not fix Case 2!
• a single left rotation also does not fix Case 3

58
Left-right rotation for Case 2

• left-right double rotation a left rotation of
  the left child, followed by a right rotation at
  the parent

59
Left-right rotation, steps

• perform left-rotate on left child
• perform right-rotate on parent (current node)

60
Left-right rotation example
61
Right-left rotation for Case 3

• right-left double rotation a right rotation of
  the right child, followed by a left rotation at
  the parent

62
Right-left rotation, steps

• perform right-rotate on right child
• perform left-rotate on parent (current node)

63
AVL tree practice problem

• Draw the AVL tree that would result if the
  following values were added in this order to an
  initially empty tree
• Harry, Hermione, Ron, Neville, Padma
• As you add each node to the tree, what is the
  balance factor at each node? What rotation is
  needed, if any?

64
AVL tree more practice

• Draw the AVL tree that would result if the
  following values were added in this order to an
  initially empty tree
• TJ, Matt, Brenton, Emily, Chris, Jake, Paige,
  Lee, Piper, Amber, Morgan
• As you add each node to the tree, what is the
  balance factor at each node? What rotation is
  needed, if any?

65
Building AVL Trees

• After normal BST add, update heights from new
  leaf up towards root
• If balance factor changes to gt 1 or lt -1, then
  use rotation(s) to rebalance
• Let n be the first unbalanced node found
• Case 1 n has balance factor -2 and n's left
  child has balance factor of 1
• fixed by performing right-rotation on n
• Case 2 n has balance factor -2 and n's left
  child has balance factor of 1
• fixed by perform left-rotation on n's left child,
  then right-rotation on n (left-right double
  rotation)

66
Building AVL Trees

• Case 3 n has balance factor 2 and n's right
  child has balance factor of 1
• fixed by perform right-rotation on n's right
  child, then left-rotation on n (right-left double
  rotation)
• Case 4 n has balance factor 2 and n's right
  child has balance factor of 1
• fixed by performing left-rotation on n
• After rebalancing, continue up the tree updating
  heights