Analysis of Algorithms

1
Analysis of Algorithms

• Amortized Analysis
• Aggregate Analysis
• The Accounting Method
• The Potential Method
• An Application Dynamic Tables

2
Amortized Analysis

• This covers a collection of methods that provide
  deterministic upper bounds for sequences of
  operations where worst case analysis applied to
  each operation might result in upper bounds that
  are not representative (overly pessimistic) of
  the actual cost of the sequence.
• Examples and applications
• Stack Management with MULTIPOP.
• INCREMENT and DECREMENT of a binary counter.
• Dynamic Tables - e.g., for hashing applications.

3
Amortized Analysis

• Aggregate Analysis we find the worst case time
  T(n) for a sequence of n operations. The
  quotient T(n)/n will give us the average (or
  amortized) worst-case cost per operation.
  Although each operation could have a large
  individual cost, the number of operations will
  limit the possible maximum cost per operation.
• Ex. Stack operations.
• Push(S, x)
• Pop(S)
• Multipop(S, k)
• While not Stack-Empty(S) and k ? 0
• do Pop(S)
• k lt-- k - 1

4
Amortized Analysis

• The worst case for a Multipop(S, n) is O(n), so a
  naïve approach would say that the worst case cost
  for n operations is nO(n) O(n2).
• BUT you cant pop want you did not push.
• The total number of pushes and pops over n
  operations cannot be more than n. Some of the
  Pops will occur in the context of the Multipop,
  others alone. We can allocate an O(1) cost to Pop
  or Multipop on an empty stack Push has a
  constant cost of O(1). Pop has a cost of O(1),
  regardless of the state of the stack.
  Multipop(S, k) has a cost of O(min(s, k)), where
  s S.
• The total cost for n operations cannot exceed
  O(n). The average cost per operation is then
  O(n)/n O(1).

5
Amortized Analysis

• Incrementing a Binary Counter.
• Let A0, 1, , k-1 be an array of bits,
  length(A) k the binary numbers (x) are stored
  with their lowest order bit in A0
  . Initially, x 0.
• Increment(A)
• i lt-- 0
• while i lt lengthA and Ai 1
• do Ai lt-- 0
• i lt-- i 1
• if i lt lengthA
• then Ai lt-- 1

6
Amortized Analysis

• The worst case occurs when all bits are 1 except
  for the highest, which is 0. In that case
  Increment(A) has a cost .
• n operations would thus have a cost .
• This is naïve some operations involve only ONE
  bit flipping, some involve all non-zero bits
  flipping. Can we do better?
• Observe
• A0 flips every time Increment(A) is called.
• A1 flips every second time Increment(A) is
  called.
• A2 flips every fourth time
• Ai flips times in a sequence of n
  Increments, where i 0, 1, , .

7
Amortized Analysis

• The total number of flips in the sequence is
  given by
• The worst case for a sequence of n Increments is
  O(n), and the amortized cost per operation is
  O(n)/n O(1).
• Note look at Building a heap in the Heapsort
  chapter for a similar argument in the proof that
  heaps can be built in linear time.

8
Amortized Analysis

• The Accounting Method.
• We assign different charges to different
  operations - sometimes more than appropriate,
  sometimes less.
• The amount we charge amortized cost.
• When charged more than the actual cost, an
  operation will save some credit when charged
  less, it will have to draw down some of the
  accumulated credit.
• The Stack Example.
• Operation Charge
• Push 2
• Pop 0
• Multipop 0

9
Amortized Analysis

• When a Push is executed, we put 2 on the plate
  and take 1 to pay for the operation.
• When Pop is executed, we put nothing on the plate
  (charge 0 to the operation) and take 1, or 0 if
  the stack is empty.
• When Multipop is executed, we put nothing on the
  plate and take 1 for each Pop operation actually
  executed.
• Total cost over n operations O(n).
• Each operation has cost O(1).

10
Amortized Analysis

• The Increment Example.
• Operation Cost
• Set a bit to 1 2
• Set a bit to 0 0
• When setting a bit to 1, put two units on the
  plate and withdraw one when setting a bit to 0,
  withdraw a unit from the plate.
• Cost of n operations is O(n). Cost per operation
  O(1).

11
Amortized Analysis

• The Potential Method.
• The method involves constructing a potential
  function, which captures any prepaid amount as
  potential energy - by analogy to the physics of
  pushing bodies up the side of a hill. The energy
  will be used by some operations.
• The amortized cost of the ith operation
• where Di denotes a data structure containing all
  the information after the ith operation, and ci
  is the actual cost of the ith operation. The
  total amortized cost of the n operations

12
Amortized Analysis

• For the amortized cost to be an upper bound on
  the actual cost, we need to have
• It thus makes sense to require the potential
  function to satisfy , for all
  i.
• Defining so it is 0, and then defining
  so it is nonnegative for all other values
  of Di, will provide us with a usable potential
  function (we still have to find it, in each
  specific case).

13
Amortized Analysis

• The Stack.
• Define , where we use Si as Di.
• Cost of a Push operation (actual cost ci 1)
• Push(Si)
• Multipop(Si, k)
• where the actual cost of the operation is k
  min(k, s).
• Multipop(Si, k)
• A similar analysis leads to an amortized cost of
  Pop(S) of 0. Worst case of n operation O(n).

14
Amortized Analysis

• The Bit Counter.
• bi number of 1s in the array after
  the ith Increment(A) operation.
• Costs suppose the ith Increment operation resets
  ti bits. Its cost is then at most ti 1, since
  it will set at most one extra bit to 1.
• If bi 0, then the ith operation resets all k
  bits, so that bi-1 ti k. If bi gt 0, then bi
  bi-1 - ti 1.
• In either case bi bi-1 - ti 1, and the
  potential difference is
• The amortized cost of the operation is

15
Amortized Analysis

• Case 1 the counter starts at 0
• It is trivial to see that amortized cost of a
  sequence on n Increment operations is O(n).
• Case 2 the counter starts at something other
  than 0.
• Let b0 0 denote the number of 1s in the
  original array A0, , k-1. Let bn k denote
  the number of 1s after n Increment operations.
  The equation
• leads to the inequality
• b0  k, and, as long as k O(n), the total
  actual cost is O(n). If we execute at least
  Increments, the total cost is O(n).

16
Amortized Analysis

• Dynamic Tables.
• You have a table into which you are inserting
  items or from which you are deleting items. You
  do not know a priori how many items you will
  have, and you need to use, at any time, no more
  than twice the memory used for the items (well,
  as little extra memory as we can get away with,
  without thrashing i.e. spending a lot of time
  making and unmaking tables).
• What do you do?

17
Amortized Analysis

• Insert until the table is full.
• Create a new table double the size and copy into
  half of it the items in the full table.
• Garbage collect the old table.
• Repeat 1-3 as needed
• Delete until the table is sparse.
• If the table is 3/4ths (or 2/3rds) empty, create
  a new table of half the size of the original,
  copy (the new table is at least half full but
  not too full) into the new table.
• Garbage collect the old table.
• Repeat 5-7 as needed.
• Cost Use the three methods to analyze the costs