Algorithms

1
Algorithms

• Unit 10. Amortized Analysis
• Chih-Hung Wang

• Aggregate Analysis
• The Accounting method
• The Potential Method
• Dynamic Table

2
Amortized Analysis

• The time required to perform a sequence of data
  structure operations in average over all the
  operations performed.
• Average performance of each operation in the
  worst case.
• For all n, a sequence of n operations takes worst
  time T(n) in total. The amortized cost of each
  operation is T(n)/n.

3
Methods

• 3 methods
• aggregate analysis
• accounting method
• potential method

4
Aggregate Analysis

• Stack operation
• - PUSH(S, x)
• - POP(S)
• - MULTIPOP(S, k)

5
Multipop
6
Example
7
Analysis

• Analysis a sequence of n PUSH, POP, and MULTIPOP
  operations on an initially empty stack.
• O(n2) ?
• O(n) (better bound)
• Each object can be popped only once per time that
  its pushed.
• Have n PUSHes?n POPs, including those in
  MULTIPOP.
• Therefore, total cost O(n).
• Average over the n operations ? O(1) per
  operation on average.
• The amortized cost of an operation is O(n)/nO(1)

8
Incrementing a binary counter

• Algorithm

9
Example
10
Analysis

• O(n k) (k is the word length)
• Amortized Analysis
• A0 flip n times A1 flip ?n/2? times
• A2 flip ?n/4? times

11
The Accounting Method

• Assign different charges to different operations.
• Some are charged more than actual cost.
• Some are charged less.
• Amortized cost amount we charge.
• When amortized cost gt actual cost, store the
  difference on specific objects in the data
  structure as credit.

12
Credit

• If we denote the actual cost of the ith operation
  by ci and the amortized cost of the ith operation
  by , we require
• for all sequence of n operations.
• The total credit stored in the data structure is
  the difference between the total actual cost,
  or
• .

13
Stack Operations

• Actual cost
• PUSH 1,
• POP 1,
• MULTIPOP min(k,s).
• Amortized cost
• PUSH 2,
• POP 0,
• MULTIPOP 0.

Amortized Cost O(1)
14
Incrementing a Binary Counter

• Actual Cost
• 0 -gt 1 1
• 1 -gt 0 1
• Amortized Cost
• 0 -gt 1 2
• 1 -gt 0 0

15
Incrementing a Binary Counter

• Have 1 of credit for every 1 in the counter.
• Therefore, credit 0.
• Amortized cost of INCREMENT
• Cost of resetting bits to 0 is paid by credit.
• At most 1 bit is set to 1.
• Therefore, amortized cost 2. gt O(1)
• For n operations, amortized cost O(n).

16
The Potential Method

• Like the accounting method, but think of the
  credit as potential stored with the entire data
  structure.
• Accounting method stores credit with specific
  objects.
• Potential method stores potential in the data
  structure as a whole.
• Can release potential to pay for future
  operations.
• Most flexible of the amortized analysis methods.

17
Notations

• Let Di data structure after ith operation,
• D0 initial data structure,
• ci actual cost of ith operation,
• amortized cost of ith operation.

18
Potential Function

• Potential function Di ? R
• ?(Di) is the potential associated with data
  structure Di.

The total amortized cost of the n operations is
19
Potential Function

• If we require that ?(Di) ?(D0) for all i, then
  the amortized cost is always an upper bound on
  actual cost.
• In practice ?(D0) 0, ?(Di) 0 for all i .

20
Stack Operations

• ?(Di) the number of objects in the stack of the
  ith operation.
• ?(D0)0
• ?(Di) ?0

21
PUSH POP

• PUSH
• Multipop

22
POP Amortized Cost

• The amortized cost of an ordinary POP operation
  is 0.
• The amortized cost of each of three operations is
  O(1), and thus the total amortized cost of a
  sequence of n operations is O(n).

23
Incrementing a Binary Counter

• ?(Di) bi the number of 1s in the counter
  after the ith operations.
• The ith INCREMENT operation resets ti bits.
• The Counter starts at zero
• ?(D0)0 ?(Di) ?0
• Total amortized cost of n operations is O(n)

24
Does not Start at Zero
25
Dynamic Tables

• A nice use of amortized analysis.
• Scenario
• Have a tablemaybe a hash table.
• Dont know in advance how many objects will be
  stored in it.
• When it fills, must reallocate with a larger
  size, copying all objects into the new larger
  table.
• When it gets sufficiently small, might want to
  reallocate with a smaller size.
• Details of table organization not important.

26
Dynamic Tables

• Goals
• 1. O(1) amortized time per operation.
• 2. Unused space always constant fraction of
  allocated space.
• Load factor a num/size,
• num items stored
• size allocated size.
• If size 0, then num 0. Call a 1.
• Never allow a gt 1.
• Keepa gt a constant fraction ?goal (2).

27
Table Expansion

• Consider only insertion.
• When the table becomes full, double its size and
  reinsert all existing items.
• Guarantees that a 1/2.
• Each time we actually insert an item into the
  table, its an elementary insertion.

28
Table Expansion
29
Analysis

• Running time Charge 1 per elementary insertion.
  Count only elementary insertions, since all other
  costs together are constant per call.
• ci actual cost of ith operation
• If not full, ci 1.
• If full, have i - 1 items in the table at the
  start of the ith operation. Have to copy all i -
  1 existing items, then insert ith item?ci i .
• n operations ?ci O(n)? O(n2) time for n
  operations. (?)

30
Aggregate Method
Amortized cost 3
31
Accounting Method

• Each item pays for 3 elementary insertions
• 1. inserting itself in the current table,
• 2. moving itself when the table is expanded, and
• 3. moving another item that has already been
  moved once when the table is expanded.
• Charge 3 per insertion of x.
• 1 pays for xs insertion.
• 1 pays for x to be moved in the future.
• 1 pays for some other item to be moved.

32
Accounting Method

• Suppose weve just expanded, size m before
  expansion, size 2m after expansion.
• Assume that the expansion used up all the credit,
  so that theres no credit stored after the
  expansion.
• Will expand again after another m insertions.
• Each insertion will put 1 on one of the m items
  that were in the table just after expansion and
  will put 1 on the item inserted.
• Have 2m of credit by next expansion, when there
  are 2m items to move. Just enough to pay for the
  expansion, with no credit left over!

33
Potential Method
(No expansion)
34
Potential Method
(Expansion)
35
Figure
36
Table Expansion and Contraction

• To implement a TABLE-DELETE operation, it is
  desirable to contract the table when the load
  factor of the table becomes too small, so that
  the waste space is not exorbitant.

37
Goal

• The load factor of the dynamic table is bounded
  below by a constant.
• The amortized cost of a table operation is
  bounded above by a constant.
• Set load factor ? ½.

38
Solution

• The first n/2 operations are inserted .
  The second n/2 operations, we perform
• I, D, D, I, I, D, D,
• Total cost of these n operations is .
  Hence the amortized cost is .
• Set load factor ? ¼ (as TABLE_DELETE)
• (after the contraction, the load factor
  become ½ )

39
Potential Method Analysis
40
Figure
41
TABLE-INSERT

• if , same as before.
• if
• if

42

• If

43
TABLE-DELETE (1)
does not cause a contraction (i.e.,
44
TABLE-DELETE (2)
causes a contraction
45
TABLE-DELETE (3)