Minimum Cost Binary Trees

1
Minimum Cost Binary Trees
Speaker Dana Moshkovitz
2
Outline

• Minimum Cost Binary Trees The problems
  description
• The Garsia-Wachs algorithm
• Kingstons proof for the Garsia-Wachs algorithm

3
Minimum Cost Binary Trees Description
Given a list of n weights (nonnegative real
numbers), p1,...,pn, let us look at the class of
binary trees, which these weights are assigned to
their leaves in the same order given. we would
like to find the tree with the minimum weighted
external path length, i.e. that tree T, s.t
is minimal (hi, namely the level of pi, is
the number of arcs in T along the path from the
root to the leaf, whose weight is pi)
Lets observe an example...
4
Minimum Cost Binary Trees Presenting the problem
Example
Suppose these are the weights we get
We can construct the following tree
Now let us add to each leave its weighted
external path length
Total 106
20
14
6
12
36
18
5
Minimum Cost Binary Trees Presenting the problem
Example
6
Minimum Cost Binary Trees Solving the
problem Intuition
Our first observation is that the smaller the
weight, the lower it should be in the tree. The
above obvious observation is all we needed in the
unordered version. Let us recall Huffmans
simple greedy algorithm. The general idea behind
his solution was to construct the tree from the
bottom and always choose the two smallest weights
to be siblings. Let us demonstrate...
7
Minimum Cost Binary Trees Solving the
problem Intuition - Back to Huffmans Algorithm
Again we have these weights
We choose the two less expensive fatherless
weights and create them a father
Total 100
8
Minimum Cost Binary Trees Solving the
problem Intuition
The question arising here is whether and how can
we apply the same idea to the ordered case. The
problem now is that we cannot arbitrarily choose
the two cheapest leaves. The two leaves we choose
must be neighbors. Unfortunately, this demand
makes the naive greedy algorithm incorrect (For
example, suppose the weights are 4,3,4,4. The
correctness of the result is depended on whether
we choose lt4,3gt or lt3,4gt to be the minimal
pair). So, what do we do??
9
Minimum Cost Binary Trees Solving the
problem Toward a Solution

• The feel is that we have some pretty useful
  ideas
• Constructing the tree from the bottom
• Finding siblings in each iteration
• However, as we have already seen, they still need
  some refinement.

10
Minimum Cost Binary Trees Solving the
problem Toward a Solution
We need an additional observation Suppose we
have weights p1,...,pn and levels h1,...,hn.
There is only one binary tree (at most)
determined by this data, and that tree can be
computed efficiently. Lets demonstrate and
explain this
11
Minimum Cost Binary Trees Solving the
problem Toward a Solution
Given feasible weights p1,...,pn and levels
h1,...,hn, let us see how can we construct the
binary tree they represent.
Algorithmically we can obtain this by preserving
an array, which will hold in its i-th entry a
pointer to a node in the i-th level lacking a
right son.
12
Minimum Cost Binary Trees Solving the
problem Toward a Solution
We conclude, that it suffices to solve the
unordered version under the constraint that the
level of each leaf in the resulting tree equals
the level of the corresponding leaf of an ordered
solution. What we need to explain now is how do
we solve the unordered problem under this
constraint. The question is how do we choose the
right pair of neighbors.
13
Minimum Cost Binary Trees Solving the
problem Right Minimal Pairs
Remark From now on, we shall treat our list of
weights as though it consists of additional two
sentinels, namely ?and ?1, placed in the
leftmost and rightmost positions in the list
respectively. This is done for the ease of the
discussion, proving the modification does not
alter the solution is rather trivial (see Gilbert
and Moores result).
Let us define the following A pair of leaves
pi-1,pi is right minimal (briefly R.M.) if (i)
1lti?n (ii) pi-2pi-1 ? pi-1pi (iii) pi-1pi lt
pj-1pj for all jgti
In other words, two neighbors are right minimal
if their sum is minimal among the sums to their
right, but this does not hold for the pairs to
their left.
14
Minimum Cost Binary Trees Solving the
problem Some Interesting Facts about R.M. Pairs
Lemma 1 Suppose we have a sequence of at least
three weights pa,pa1,...,pb, s.t. pj-1 pjlt pj
pj1, for altjltb, then ha? ha1?...? hb-1 in every
minimal tree containing these weights.
15
Minimum Cost Binary Trees Solving the
problem Some Interesting Facts about R.M. Pairs
Proof. Suppose hj-1lthj for some altjltb. pj must be
a left child. Then the transformation
is both legal and less expensive since pj1 must
be in R, R?pj1gtpj-1. Therefore the lemma
holds. ?
16
Minimum Cost Binary Trees Solving the
problem Some Interesting Facts about R.M. Pairs
Lemma 2 if pi-1,pi is the rightmost R.M. pair,
then hi-1? hi?...? hn in every minimal tree.
Proof. Directly follows from the choice of the
pair and from lemma 1. ? Further explanation
notice that when a pair is the rightmost R.M
pair, the sequence of pairs to its right
constructs a monotone series. (Otherwise the pair
that breaks the monotony would be the rightmost
R.M pair).
17
Minimum Cost Binary Trees Solving the
problem Some Interesting Facts about R.M. Pairs
Lemma 3 if pi-1,pi is the rightmost R.M. pair,
then hi-1 hi in some minimal tree. Proof. By
lemma 2 it suffices to show that hi-1? hi in
some minimal tree. Suppose we have a minimal tree
such that hi-1gthi . In this tree pi-1 is a right
child, so we can use the following transformation
to get another minimal tree
Again, R? pi-2? pi, so the new tree is
necessarily minimal. ?
18
Minimum Cost Binary Trees Solving the problem ...
And finally...
After we clarified the most important notion for
the purpose of this lecture, namely R.M. pairs,
and proved some interesting properties related to
such, let us finally observe the desired
algorithm. Afterwards we shall prove the
correctness of our solution.
19
Minimum Cost Binary Trees Solving the problem The
Algorithm
Execute the following two steps n-1 times (1)
Locate the rightmost R.M pair of entries. (2)
Find the first entry to its right, which is
greater than/equals it, and move the pair to the
left of this entry.
20
Minimum Cost Binary Trees Solving the
problem Simulating the algorithm
We start with the same input
The heights yield the tree we have already seen
We locate the rightmost R.M pair, pair them and
move them to their right place.
21
Minimum Cost Binary Trees Proving the Solution
Lemma 4 Let pik1 be the first node to the
right of the rightmost R.M pair pi-1,pi, s.t
pik1gt pi-1pi. In some minimal tree T for which
hi-1 hi, either (a) hik hi-1,or (b) hik
hi, and pik is a right child. Proof. Begin with
T the minimal tree of lemma3. By lemma 2 we
merely need to show that hik? hi-1 in T. Let us
suppose hiklt hi-1. Let pm (m?ik) be the first
node to the right of pi, s.t. hmlt hi-1.
pmltpi-1pi.
22
Minimum Cost Binary Trees Proving the Solution
There are two possibilities (a) pi-1,pi are
siblings in T (b) pi,pi1 are siblings in T Let
p,q denote the two siblings. We can transform T
as follows and improve the cost by decreasing
pq-pm? pi-1pi-pmgt0. This contradicts the
hypothesis T was minimal. Hence hik? hi-1.
23
Minimum Cost Binary Trees Proving the Solution
It remains to prove, that when hik hi, pik can
be made a right child. If pik is already a right
child, we are done. Otherwise, if kgt0, we can
transform T
24
Minimum Cost Binary Trees Proving the Solution
While if k0, we can use this transformation
pik1
pi-1
pi
pik1
pi-1
pi
In both cases, our claim holds. ?
25
Minimum Cost Binary Trees
Proving the solution
Lemma 5 For every rearrangement done in an
iteration of the algorithm, the cost for the new
series cannot exceed the cost for the original
series of weights. Proof. Let k be the number of
places we move the R.M. pair in a specific
iteration. We will exhibit a tree T for the new
arrangement, whose leaves has the same levels as
in T, the minimum tree of lemma 4. This -
of-course - will prove the lemma. If k0, pi-1
and pi are siblings in T, so we may take TT.
26
Minimum Cost Binary Trees
Proving the solution
Otherwise kgt0. First we will transform T, so pi-1
and pi (the current rightmost R.M. pair) are
siblings (If they they are not so already)
Lemma 4 states, that they have the same level
Lemma 4 states pi1 is a right child
27
Minimum Cost Binary Trees
Proving the solution
Since kgt0, the first weight greater than/equal to
pi-pi, namely pik1, is still to the right of
pi. By lemma 4, there are merely two
possibilities hikhi or hikhi-1. In each case
all we need to do is slid the new node to the
right until it passes pik. ?
28
Minimum Cost Binary Trees
Proving the solution
Lemma 6 For every rearrangement done in an
iteration of the algorithm, the cost for the new
series is at least the cost for the original
series of weights, and if equality holds, then
the two solutions have corresponding levels.
Proof. Now - given a minimum tree T for the new
arrangement - we need to construct a minimum tree
T for the original series (while preserving the
levels). Again we denote the number of places we
move the pair by k. If k0, TT.
29
Minimum Cost Binary Trees
Proving the solution
Otherwise kgt0. We know that pikltpi-1pi?pik1,
so we can use lemma 1 in T with
pi1,...,pik,px, pik1 (px is the father of the
R.M. pair) to obtain hi1?... ?hik?hx in T.
Hence, either
or
In the first case the treatment is simple we
need to move Tx to the left.
30
Minimum Cost Binary Trees
Proving the solution
The treatment of the second case consists of two
phases (a) Slid Tx to the left across all the
leaves in its level. (b) Rotate all the nodes two
places to the right.
31
Minimum Cost Binary Trees
Proving the solution
How do we perform the rotation? A simulation
Tx
pi-1
pi
pi-1
pi1
pi1
pi
pi1
pi
pi-1
32
Minimum Cost Binary Trees
Proving the solution
For every level the pair drops downward (possibly
0), some other pair moves up one level. But we
have taken a R.M. pair (for which all the pairs
with greater indices weight more than it does).
This implies that w(T)w(Ta)?w(T), so the weight
of T is at least the weight of a minimum cost
tree for the original series of weights. If
equality holds, the level of the pair is not
changed by the rotation, so T is a minimum cost
tree for the original series, which preserves the
levels of T.?
33
Minimum Cost Binary Trees
Proving the solution
Those two final lemmas allow us to finally state
the following The algorithm we presented is
correct. ?
34
Minimum Cost Binary Trees Solving the
problem Another way to view this algorithm
We start with the same input
Which yield the tree we have already seen
We locate the rightmost R.M pair, pair them and
move them to their right place.
35
Minimum Cost Binary Trees References
This overview was based on 1 J. H. Kingston. A
new proof for the Garcia-Wachs algorithm. J. of
Algorithms, 9129-136,1988 The Garsia-Wachs
algorithm 2 A. M. Garsia and M. L. Wachs, A
new algorithm for minimum cost binary trees.
Sicomp, 6(4), 622-642, 1977 Huffmans
algorithm 3 D. Huffman, A method for the
construction of minimum- redundancy codes, Proc.
IRL, 1098-1101, 1952
36
Minimum Cost Binary Trees References
You might be interested in previous results
concerning this problem The original O(n3)
algorithm 4 E. N. Gilbert and E.F. Moore,
Variable length binary encodings, Bell Systems
Tech, 38, 933-968, 1977. An O(n2)
algorithm 5 D. E. Knuth, Optimum binary search
trees, Acta Inform. 1, 14-25, 1971. The
Hu-Tucker algorithm (O(nlogn)) 6 T.C. Hu and
C. Tucker, Optimum computer search trees, SIAM J.
Appl. Math., 514-532, 1971. 7 T.C. Hu, A new
proof of the T-C algorithm, SIAM J. Appl. Math.
25, 83-94, 1973.