15-211 Fundamental Structures of Computer Science

1
15-211Fundamental Structuresof Computer Science
Binary Search Trees
Ananda Guna

• Jan. 23, 2003

Based on lectures given by Peter Lee, Avrim Blum,
Danny Sleator, William Scherlis, Ananda Guna
Klaus Sutner
2
First a Review of Stacks and Queues
3
A Stack interface
public interface Stack public void
push(Object x) public void pop() public
Object top() public boolean isEmpty()
public void clear()
4
Stacks are LIFO
Push operations
e
d
c
b
a
5
Stacks are LIFO
Pop operation
e
Last element that was pushed is the first to be
popped.
d
c
b
a
6
A Queue interface
public interface Queue public void
enqueue(Object x) public Object dequeue()
public boolean isEmpty() public void clear()
7
Queues are FIFO
back
front
k
r
q
c
m
8
Queues are FIFO
Enqueue operation
back
front
k
r
q
c
m
y
9
Queues are FIFO
Enqueue operation
front
back
k
r
q
c
m
y
10
Queues are FIFO
Dequeue operation
front
back
k
r
q
c
m
y
11
Implementing stacks, 1
Linked representation. All operations constant
time, or O(1).
12
Implementing stacks, 2

• An alternative is to use an array-based
  representation.
• What are some advantages and disadvantages of an
  array-based representation?

a
b
c
top
13
A queue from two stacks
Enqueue
Dequeue
j
a
What happens when the stack on the right becomes
empty?
i
b
h
c
g
d
f
e
14
Now to Trees
15
(No Transcript)
16
CS is upside down
root
leaves
17
Trees are everywhere

• Trees are everywhere in life.
• As a result, in computer programs, trees turn out
  to be one of the most commonly used data
  structures.

18
Arithmetic Expressions


5
2
7
19
Game trees
20
Directory structure
/afs
cs
andrew
acs
course
usr
usr
15
18
127
211
21
Tree Definitions

• A tree is a set of nodes and a set of directed
  edges that connects pairs of nodes.
• A tree is a a Directed, Acyclic Graph (DAG) with
  the following properties
• - one vertex is distinguished as the
  root no edges enter this vertex
• - every other vertex has exactly one
  entering edge

22
Trees, more abstractly

• A tree is a directed graph with the following
  characteristics
• There is a distinguished node called the root
  node.
• Every non-root node has exactly one parent node
  (the root has none).

23
A closer look at Trees

R
siblings
T1
T3
T2
24
Unique parents
root
a
b
c
d
e
f
25
Implementation of Trees

• How do we implement a general tree? Eg A file
  system
• Each node will have two links
• One to its left most child
• One to its right sibling

26
Implementation of a binary tree with an array

• Assume that the left child of node i (i1.) is
  stored at 2i and right child of node I is stored
  at 2i1
• Draw the tree represented by the following array
  (assume indices start from 1)
• 12 10 15 8 11 14 18
• Question What is the minimum height of a binary
  tree with n nodes? What is the maximum height?

27
Binary Tree Traversals

• Inorder Left-Root-Right
• Use stack or recursion
• PreOrder Root-Left-Right
• Use Stack or recursion
• PostOrder-Left-Right-Root
• Use Stack or recursion
• Level Order Traversal
• Use a queue
• What is the output of each of the
• traversal?
• (see next slide for BFS in a tree)

28
Algorithm for Breadth-first traversal (of a tree
using a queue)

• enqueue the root
• while (the queue is not empty)
• dequeue the front element
• print it
• enqueue its left child (if
  present)
• enqueue its right child (if
  present)

29
Facts and Questions About Trees

• A path from node n1 to nk is defined as a path
  n1,n2, . nk such that ni is the parent of ni1
• Depth of a node is the length of the path from
  root to the node. What is the depth of root? What
  is the maximum depth of a tree with N nodes?
• What is the number of edges in a tree with N
  nodes?
• Height of a node is length of a path from node to
  the deepest leaf. The height of the tree is the
  ________________?
• Let T(n) be the number of null pointers in a tree
  of n nodes. Show that T(n) n 1

30
Time to think about complexity of Algorithms

• Considering algorithms
• Is the approach correct?
• How fast does it run?
• How much memory does it use?
• Can I finish writing the code in the next 8
  hours?
• What is most important?
• Consider fib(n) fib(n-1)fib(n-2) for n gt 2
• fib(0)fib(1)1
• Lets look at a simple algorithm

31
Fibonacchi Tree
Closed form

• public static long fib(int n)
• if (n lt 1) return 1
• return fib(n-1) fib(n-2)
• It turns out the number of function calls is
  proportional to fib(n) itself! In fact, it's
  exactly 2fib(n) - 1.
• fib(90) takes about 7000 years on 1Ghz machine.

F(5)
F(3)
F(4)
F(1)
F(2)
F(2)
F(3)
F(1)
F(2)
F(1)
F(0)
F(0)
F(1)
F(1)
F(0)
32
Making Fibonacci more efficient

• Can we write a better algorithm?
• Can we reuse some of the parts of the recursion?
• // call initially as fastfib(0,1,n)
• public static long fastfib(long prev, long
  current, int togo)
• if (togo lt 0) return current
• return fastfib(current, currentprev,
  togo-1)
• What is the complexity of this algorithm?

33
A question about height and number of nodes in a
binary tree

• Suppose we have n nodes in a complete binary tree
  of height h. What is the relation between n and
  h?
• The number of nodes in level i is 2i (i0,1,,h)
• Therefore total nodes in all levels is
• So what is the relation between n and h?
• A binary tree is completely full if it is of
  height, h, and has 2h1-1 nodes.

34
Bit about asymptotic analysis

• O notation
• T(n) is O(f(n)) if there exist two positive
  constants c and n0 such that T(n) lt cf(n) for
  all n gt n0
• Omega notation
• T(n) is Omega(f(n)) if there exist two positive
  constants c and n0 such that T(n) gt cf(n) for
  all n gt n0
• Theta notation
• T(n) is Theta(f(n)) if it is both O(f(n)) AND
  Omega(f(n)).

35
Big-Oh notation
c?f(N)
T(N) O(f(N)) T(N) is order f(N)
T(N)
running time
N
36
Some examples

• If f(n) 10n 5 and g(n) nshow f(n) is
  O(g(n))
• f(n) 3n2 4n 1. Show f(n) is O(n2)
• show that 5log(n) is O(n)
• f(n) 3n2 4n 1. Show f(n) is ?(n2)
• Therefore f(n) theta(n2)

37
Logarithms and exponents

• Logarithms and exponents are everywhere in
  algorithm analysis

logba c if a bc
38
Logarithms and exponents

• Usually will leave off the base b when b2, so
  for example

log 1024 10
39
Some useful equalities
logbac logba logbc logba/c logba -
logbc logbac clogba logba (logca) /
logcb (ba)c bac babc bac ba/bc ba-c
40
Big-Oh again

• When T(N) O(f(N)), we are saying that T(N)
  grows no faster than f(N).
• I.e., f(N) describes an upper bound on T(N).
• Put another way
• For large enough inputs, c?f(N) always
  dominates T(N).
• Called the asymptotic behavior

41
Big-O characteristic

• If T(N) c?f(N) then
• T(N) O(f(N))
• Constant factors dont matter
• Because of this, when T(N) O(c?g(N)), we
  usually drop the constant and just say O(g(N))

42
Big-O characteristic

• Suppose T(N) k, for some constant k
• Then T(N) O(1)

43
Big-O characteristic

• More interesting
• Suppose T(N) 20n3 10nlog n 5
• Then T(N) O(n3)
• Lower-order terms dont matter
• Question
• What constants c and n0 can be used to show that
  the above is true?
• Answer c35, n01

44
Big-O characteristic

• If T1(N) O(f(N)) and T2(N) O(g(N)) then
• T1(N) T2(N) max(O(f(N)), O(g(N)).
• The bigger task always dominates eventually.
• Also
• T1(N) ? T2(N) O(f(N) ? g(N)).

45
Some common functions
46
BST-An Inductive Perspective

• Let's focus on binary trees (left/right child
  only).
• A binary tree is either
• empty (we'll write nil for clarity), or
• looks like (x,L,R) where
• x is the element stored at the root, and
• L, R are the left and right subtrees of the
  root.

47
In Pictures
Empty Tree
48
Flattening a BT
T
flat(T) e, b,f,a,d,g
49
Def Binary Search Tree
A binary T is a binary search tree (BST)
iff flat(T) is an ordered sequence.
Equivalently, in (x,L,R) all the nodes in L
are less than x, and all the nodes in R are
larger than x.
50
Example
flat(T) 2,3,4,5,6,7,9
51
Why do we care?
versus
52
Binary Search
How does one search in a BST?
search(x,nil) false search(x,(x,L,R))
true search(x,(a,L,R)) search(x,L)
xlta search(x,(a,L,R)) search(x,R) xgta
53
Correctness
Clearly, search() can never return a false
positive answer. But search() only walks down
one branch, so how do we know we don't get false
negative answers? Suppose T is a BST that
contains x. Claim search(x,T) properly
returns "true".
54
Proof
T cannot be nil, so suppose T (a,L,R). Case
1 x a done. Case 2 x lt a Since T is a
BST, x must be in L. But by induction (on
trees), search(x,L) returns true. Done. Case 3
x gt a same as case 2.
55
Insertions
Insertions in a BST are very similar to
searching find the right spot, and then put
down the new element as a new leaf. We will
not allow multiple insertions of the same
element, so there is always exaxtly one place for
the new guy.
56
How Many?
How many decisions do we have to make before we
have either found the element, or know it's not
in the tree? We walk down a branch in the tree,
so the worst case RT for search is O( depth
of T ) O( nodes )
57
Good Tree
But in a "good" BST we have depth of T O( log
nodes )
Theorem If the tree is constructed from n inputs
given in random order, then we can expect the
depth of the tree to be log2 n. But if the
input is already (nearly, reverse,) sorted we
are in trouble.
58
Forcing good behavior
It is clear (?) that for any n inputs, there
always is a BST containing these elements of
logarithmic depth. But if we just insert the
standard way, we may build a very unbalanced,
deep tree. Can we somehow force the tree to
remain shallow? At low cost?
59
AVL-Trees

• G. M. Adelson-Velskii and E. M. Landis, 1962

1 or less
60
Next Week
More about AVL trees on tuesday Homework 1 is
due Monday 27th. This is a good time to catch up
with Java deficiencies, if any.