Transform and Conquer

1
Transform and Conquer

• Solve problem by transforming into
• a more convenient instance of the same problem
  (instance simplification)
• Presorting, Gaussian elimination, matrix
  inversion, determinant computation
• a different representation of the same instance
  (representation change)
• balanced search trees, heaps and heapsort,
  polynomial evaluation by Horners rule, Fast
  Fourier Transform
• a different problem altogether (problem
  reduction)
• reductions to graph problems, linear programming

2
Instance simplification - Presorting

• Solve instance of problem by transforming into
  another simpler/easier instance of the same
  problem
• Presorting
• Many problems involving lists are easier when
  list is sorted.
• element uniqueness
• computing the mode
• finding repeated elements
• searching
• computing the median (selection problem)

3
Selection Problem

• Find the k-th smallest element in A1,An.
• minimum k 1
• maximum k n
• median k n/2
• Presorting-based algorithm
• sort list
• return Ak
• Partition-based algorithm (decrease conquer)
• pivot/split at As using partitioning algorithm
• if sk return As
• else if sltk repeat with sublist As1,An.
• else if sgtk repeat with sublist A1,As-1.

4
Notes on Selection Problem

• Presorting-based algorithm O(nlgn) T(1)
  O(nlgn)
• Partition-based algorithm (decrease conquer)
• worst case T(n) T(n-1) (n1) -gt T(n2)
• best case T(n)
• average case T(n) T(n/2) (n1) -gt T(n)
• Bonus also identifies the k smallest elements
• Special cases max, min better, simpler linear
  algorithm (brute force)
• Conclusion Presorting does not help in this
  case.

5
Finding repeated elements

• Presorting-based algorithm
• use mergesort (optimal) T(nlgn)
• scan array to find repeated adjacent elements
  T(n)
• in total it makes T(nlgn)
• Brute force algorithm T(n2)
• Conclusion Presorting yields significant
  improvement

6
Checking element uniqueness

• Brute force algorithm T(n2)
• Algorithm PresortedElementUniqueness
• Sort the array A
• for i ? 0 to n-2 do
• if AiAi1 return false
• else return true
• Conclusion Presorting again improves
• Similar improvement for mode

7
Checking mode

• Mode is the most often met element
• Brute force scan the list and compute the
  frequencies. Then find the largest frequency
• Algorithm PresortedMode
• Sort the array A
• i ? 1 modefrequency ? 0
• while i ? n-1 do
• runlength?1 runvalue?Ai
• while irunlengthn-1 and Airunlengthrunvalu
  e
• runlength ? runlength1
• if runlengthgt modefrequency
• modefrequency?runlength, modevalue?runvalue
• i?irunlength
• return modevalue
• Conclusion Presorting again improves

8
Gaussian Elimination

• Given a system of two linear equations with two
  unknowns
• a11x a12y b1
• a21x a22y b2
• It has a unique solution unless the coefficients
  are proportional
• Express one variable as function of the other and
  substitute to solve one equation.
• What if the system has n equations and n unknowns
  ?

9
Gaussian Elimination (2)

• Transform Axb to Axb, where A upper
  triangular
• Then, solution is possible with backward
  substitution
• Elementary operations
• Exchange equations
• Replace an equation with a nonzero multiple
• Replace an equation with a sum or difference of
  this equation and some multiple of another
  equation
• Example
• 2x1 - x2 x3 1
• 4x1 x2 - x3 5
• x1 x2 x3 0

10
Gaussian Elimination (3)

• Algorithm GaussElimination
• for i ? 1 to n do Ai,n1?bi
• for i? 1 to n-1 do
• for j?i1 to n do
• for k?i to n1 do
• Aj,k?Aj,k-Ai,kAj,i/Ai,i
• Potential problems if Ai,i is zero or very small

11
Gaussian Elimination (partial pivoting)

• Algorithm GaussElimination2
• for i ? 1 to n do Ai,n1?bi
• for i? 1 to n-1 do
• pivotrow?i
• for j?i1 to n do
• if Aj,igtApivot,i pivotrow?j
• for k?i to n1 do
• swap(Ai,k,Apivotrow,k)
• for j?i1 to n do
• temp?Aj,i/Ai,i
• for k?I to n1 do
• Aj,k?Aj,k-AI,ktemp
• Efficiency

12
LU decomposition

• Byproduct of Gaussian Elimination
• Example ALU
• LUxb. Denote yUx ? Lyb
• Solve Lyb, then solve Uxy
• Solve with as many times with different bs.
• No extra space

1 0 0
2 1 0
1/2 1/2 1
2 -1 1
0 3 -3
0 0 2
13
Computing a matrix inverse

• AA-1I
• A singular matrix does not have an inverse
• A matrix is singular if and only if one of the
  rows is a linear combination of the other rows.
• Apply Gaussian elimination. If it yields an
  upper-triangular with no zeros on the diagonal,
  then the matrix is not singular
• Axjej

14
Computing the determinant

• A well-known recursive formula
• What if n is large ? Efficiency ?
• Apply Gaussian elimination.
• The determinant of an upper-triangular matrix is
  the product of elements on its diagonal.
• Efficiency ?
• Cramers rule

15
Taxonomy of Searching Algorithms

• Elementary searching algorithms
• sequential search
• binary search
• binary tree search
• Balanced tree searching
• AVL trees
• red-black trees
• multiway balanced trees (2-3 trees, 2-3-4 trees,
  B trees)
• Hashing
• separate chaining
• open addressing

16
Balanced trees AVL trees

• For every node, difference in height between left
  and right subtree is at most 1
• AVL property is maintained through rotations,
  each time the tree becomes unbalanced
• lg n h 1.4404 lg (n 2) - 1.3277
  average 1.01 lg n
  0.1 for large n
• Disadvantage needs extra storage for maintaining
  node balance
• A similar idea red-black trees (height of
  subtrees is allowed to differ by up to a factor
  of 2)

17
AVL tree rotations

• Small examples
• 1, 2, 3
• 3, 2, 1
• 1, 3, 2
• 3, 1, 2
• Larger example 4, 5, 7, 2, 1, 3, 6
• See figures 6.4, 6.5 for general cases of
  rotations

18
General case single R-rotation
19
Double LR-rotation
20
Balance factor

• Algorithm maintains balance factor for each node.
  For example

21
Heapsort

• Definition
• A heap is a binary tree with the following
  conditions
• it is essentially complete
• The key at each node is keys at its children

22
Definition implies

• Given n, there exists a unique binary tree with n
  nodes that is essentially complete, with h lg n
• The root has the largest key
• The subtree rooted at any node of a heap is also
  a heap

23
Heapsort Algorithm

• Build heap
• Remove root exchange with last (rightmost) leaf
• Fix up heap (excluding last leaf)
• Repeat 2, 3 until heap contains just one node.

24
Heap construction

• Insert elements in the order given breadth-first
  in a binary tree
• Starting with the last (rightmost) parental node,
  fix the heap rooted at it, if it does not satisfy
  the heap condition
• exchange it with its largest child
• fix the subtree rooted at it (now in the childs
  position)
• Example 2 3 6 7 5 9

25
Root deletion

• The root of a heap can be deleted and the heap
  fixed up as follows
• exchange the root with the last leaf
• compare the new root (formerly the leaf) with
  each of its children and, if one of them is
  larger than the root, exchange it with the larger
  of the two.
• continue the comparison/exchange with the
  children of the new root until it reaches a level
  of the tree where it is larger than both its
  children

26
Representation

• Use an array to store breadth-first traversal of
  heap tree
• Example
• Left child of node j is at 2j
• Right child of node j is at 2j1
• Parent of node j is at j /2
• Parental nodes are represented in the first n
  /2 locations

9
5
3
1
4
2
27
Bottom-up heap construction algorithm
28
Analysis of Heapsort

• See algorithm HeapBottomUp in section 6.4
• Fix heap with problem at height j 2j
  comparisons
• For subtree rooted at level i it does 2(h-i)
  comparisons
• Total for heap construction phase

h-1
S
2(h-i) 2i 2 ( n lg (n 1)) T(n)
i0
nodes at level i
29
Analysis of Heapsort (continued)

• Recall algorithm
• Build heap
• Remove root exchange with last (rightmost) leaf
• Fix up heap (excluding last leaf)
• Repeat 2, 3 until heap contains just one node.

T(n)
T(log n)
n 1 times
Total T(n) T( n log n) T(n log n)

• Note this is the worst case. Average case also
  T(n log n).

30
Priority queues

• A priority queue is the ADT of an ordered set
  with the operations
• find element with highest priority
  
• delete element with highest priority
  
• insert element with assigned priority
• Heaps are very good for implementing priority
  queues

31
Insertion of a new element

• Insert element at last position in heap.
• Compare with its parent and if it violates heap
  condition exchange them
• Continue comparing the new element with nodes up
  the tree until the heap condition is satisfied
• Example
• Efficiency

32
Bottom-up vs. Top-down heap construction

• Top down Heaps can be constructed by
  successively inserting elements into an
  (initially) empty heap
• Bottom-up Put everything in and then fix it
• Which one is better?

33
Horners rule

• Horner published in early 19th century
• According to Knuth, the method was used by Newton
• Evaluate a polynomial at a point x
• p(x) anxn an-1xn-1 a1x a0
• p(x) ( (anx an-1) x )x a0
• Example evaluate p(x)2x4-x33x2x-5 at x3
• p(x) x (x (x (2x-1) 3) 1) - 5
• Visualization by a table

34
Horners rule 2

• Algorithm Horner(P0..n,x)
• // Evaluate polynomial at a given point
• // Input an array P0..n of coefficients and a
  number x
• //Output the value of polynomial at point x
• p ? Pn
• for i ? n-1 down to 0 do
• p ? xp Pi
• return p
• Efficiency ?
• Byproduct coefficients of the quotient of the
  division of p(x) by (x-x0)

35
Binary exponentiation

• Horner is not efficient to compute p(x)xn at xa
• Degenerate to brute force
• Let the binary representation nblbl-1 bi b1b0
• p(x) blxl bl-1xl-1 b1x b0 and x2
• Algorithm LeftRightBinaryExponentiation
• product ? a
• for i ? l-1 down to 0 do
• product ? product product
• if bi?1 then product ? producta
• return product
• Example compute a13, n131101
• Efficiency

36
Binary exponentiation (2)

• Compute an
• Consider n bl2l bl-12l-1 b12 b0
• and multiply independent powers terms
• Algorithm RightLeftBinaryExponentiation
• term ? a
• if b01 then product ? a
• else product ? 1
• for i? 1 to l do
• term ? term term
• if bi 1 then product ? product term
• return product
• Example compute a13, n131101
• Efficiency

37
Least common multiple

• lcm(24,60)120, lcm(11,5)55
• Example 24 2 x 2 x 2 x 3
• 60 2 x 2 x 3 x 5
• lcm(24,60) (2x2x3) x 2 x 5
• Efficiency (a list of primes is required)
• lcm(m,n) mn / gcd(m,n)

38
Counting paths in a graph

• The number of different paths of length kgt0 from
  node i to node j equals the (i,j) element of the
  Ak, where A the adjacency matrix
• Example
• Efficiency

39
Reduction to graph problems

• Applies for a variety of games and puzzles
• Build the state-space graph
• Example peasant, wolf, goat, cabbage
• Traverse the graph by applying what?