??????9 : Integers, Algorithms

1
??????9 Integers, Algorithms Orders

• ??????? (Kuang-Chi Chen)
• chichen6_at_mail.tcu.edu.tw

2
Mini Review Methods
3
Useful Congruence Theorems

• Theorem 1 Let a, b?Z, m?Z. Then a?b (mod m)
  ? ?k?Z abkm.
• Theorem 2 Let a, b, c, d?Z, m?Z. Then if a?b
  (mod m) and c?d (mod m), then
• . ac ? bd (mod m), and
• . ac ? bd (mod m)

4
Integers and Algorithms
5
Integers Algorithms

• Topics
• Euclidean algorithm for finding GCDs.
• Base-b representations of integers.
• - Especially binary, hexadecimal, octal.
• - Also Twos complement representation of
  negative numbers.
• Algorithms for computer arithmetic
• - Binary addition, multiplication, division.

6
Euclids Algorithm for GCD

• Finding GCDs by comparing prime factorizations
  can be difficult when the prime factors are not
  known!
• Euclid discovered For all ints. a, b, gcd(a, b)
  gcd((a mod b), b).
• Sort a, b so that agtb, and then (given bgt1) (a
  mod b) lt a, so problem is simplified.

Euclid325-265 B.C.
7
Euclids Algorithm Example

• gcd(372,164) gcd(372 mod 164, 164).
• - 372 mod 164 372?164372/164 372?1642
  372?328 44.
• gcd(164, 44) gcd(164 mod 44, 44).
• - 164 mod 44 164?44164/44 164?443
  164?132 32.
• gcd(44, 32) gcd(44 mod 32, 32) gcd(12, 32)
  gcd(32 mod 12, 12) gcd(8, 12) gcd(12 mod 8,
  8) gcd(4, 8) gcd(8 mod 4, 4) gcd(0, 4) 4.

8
Euclids Algorithm Pseudocode

• procedure gcd(a, b positive integers)
• while b ? 0 begin
• r ? a mod b a ? b b ? r end
• return a

Sorting inputs not needed b/c order will be
reversed each iteration.
Fast! Number of while loop iterationsturns out
to be O(log(max(a,b))).
9
Base-b Number Systems

• Ordinarily, we write base-10 representations of
  numbers, using digits 0-9.
• But, 10 isnt special! Any base bgt1 will work.
• For any positive integers n,b, there is a unique
  sequence ak ak-1 a1a0 of digits ailtb such that

10
Particular Bases of Interest

• Base b10 (decimal)10 digits
  0,1,2,3,4,5,6,7,8,9.
• Base b2 (binary)2 digits 0,1. (Bitsbinary
  digits.)
• Base b8 (octal)8 digits 0,1,2,3,4,5,6,7.
• Base b16 (hexadecimal)16 digits
  0,1,2,3,4,5,6,7,8,9,A,B,C,D,E,F

Hex digits give groups of 4 bits
11
Converting to Base b

• (An algorithm, informally stated.)
• To convert any integer n to any base bgt1
• 1. To find the value of the rightmost
  (lowest-order) digit, simply compute n mod b.
• 2. Now, replace n with the quotient n/b.
• 3. Repeat above two steps to find subsequent
  digits, until n is gone (0).

12
Orders of Growth
13
Orders of Growth

• For functions over numbers, we often need to know
  a rough measure of how fast a function grows.
• If f(x) is faster growing than g(x), then f(x)
  always eventually becomes larger than g(x) in the
  limit (for large enough values of x).
• Useful in engineering for showing that one design
  scales better or worse than another.

14
Orders of Growth - Motivation

• Suppose you are designing a web site to process
  user data (e.g., financial records).
• Suppose database program A takes fA(n)30n8
  microseconds to process any n records, while
  program B takes fB(n)n21 microseconds to
  process the n records.
• Q Which program do you choose, knowing youll
  want to support millions of users?

A
15
Visualizing Orders of Growth

• On a graph, asyou go to theright, the
  faster-growing function always
  eventuallybecomes thelarger one...

fA(n)30n8
Value of function ?
Increasing n ?
16
Concept of Order of Growth

• We say fA(n) 30n8 is (at most) order n, or
  O(n).
• - It is, at most, roughly proportional to n.
• fB(n) n21 is order n2, or O(n2).
• - It is (at most) roughly proportional to n2.
• Any function whose exact (tightest) order is
  O(n2) is faster-growing than any O(n) function.
• - Later well introduce T for expressing exact
  order.
• For large numbers of user records, the exactly
  order n2 function will always take more time.

17
Definition O(g), at most order g

• Let g be any function R?R.
• Define at most order g, written O(g), to be
  f R?R ?c, k ?xgtk f(x) ? cg(x).
• - Beyond some point k, function f is at most a
  constant c times g (i.e., proportional to g).
• f is at most order g, or f is O(g), or
  fO(g) all just mean that f?O(g).
• (Often the phrase at most is omitted.)

18
Points about The Definition

• Note that f is O(g) so long as any values of c
  and k exist that satisfy the definition.
• But, the particular c, k, values that make the
  statement true are not unique Any larger value
  of c and/or k will also work.
• No need to find the smallest c and k values that
  work. (Indeed, in some cases, there may be no
  smallest values!)

However, you should prove that the values you
choose do work.
19
Big-O Proof Examples

• Show that 30n8 is O(n).
• Show ?c, k ?ngtk 30n8 ? cn.
• Let c31, k8. Assume ngtk8. Thencn 31n
  30n n gt 30n8, so 30n8 lt cn.
• Show that n21 is O(n2).
• Show ?c, k ?ngtk n21 ? cn2.
• Let c2, k1. Assume ngt1. Then cn2 2n2
  n2n2 gt n21, or n21lt cn2.

20
Big-O Example, Graphically

• Note 30n8 isntless than nanywhere (ngt0).
• It isnt evenless than 31neverywhere.
• But it is less than31n everywhere tothe right
  of n8.

30n8
30n8?O(n)
Value of function ?
n
Increasing n ?
21
Useful Facts about Big O

• Big O, as a relation, is transitive f?O(g) ?
  g?O(h) ? f?O(h)
• O with constant multiples, roots, and logs...? f
  (in ?(1)) constants a, b?R, with b?0, af, f
  1-b, and (logb f)a are all O(f).
• Sums of functionsIf g?O(f) and h?O(f), then
  gh?O(f).

22
More Big-O Facts

• ?cgt0, O(cf) O(fc) O(f?c) O(f)
• f1?O(g1) ? f2?O(g2) ?
• - f1 f2 ?O(g1g2)
• - f1f2 ?O(g1g2) O(max(g1, g2))
• O(g1) if g2?O(g1) (Very
  useful!)

23
Orders of Growth - So Far

• For any g R?R, at most order g,O(g) ? f R?R
  ?c, k ?xgtk f(x) ? cg(x).
• - Often, one deals only with positive functions
  and can ignore absolute value symbols.
• f?O(g) often written f is O(g)or f O(g).
• - The latter form is an instance of a more
  general convention...

24
Order-of-Growth Expressions

• O(f) when used as a term in an arithmetic
  expression means some function f such that
  f?O(f).
• E.g., x2O(x) means x2 plus some function that
  is O(x).
• Formally, we can think of any such expression as
  denoting a set of functions x2O(x) ? g
  ?f?O(x) g(x) x2f(x)

25
Order of Growth Equations

• Suppose E1 and E2 are order-of-growth expressions
  corresponding to the sets of functions S and T,
  respectively.
• Then the equation E1E2 really means ?f?S,
  ?g?T f g or simply S?T.
• Example x2 O(x) O(x2) means ?f?O(x)
  ?g?O(x2) x2f(x) g(x)

26
Useful Facts about Big O

• ? f, g constants a, b?R, with b?0,
• - af O(f) (e.g. 3x2 O(x2))
• - fO(f) O(f) (e.g. x2x O(x2))
• Also, if f ?(1) (at least order 1), then
• - f1-b O(f) (e.g. x?1 O(x))
• - (logb f)a O(f) (e.g. log x O(x))
• - g O(fg) (e.g. x O(x log x))
• - fg ? O(g) (e.g. x log x ? O(x))
• - a O(f) (e.g. 3 O(x))

27
(No Transcript)
28
Definition ?(g), exactly order g

• If f?O(g) and g?O(f), then we say g and f are of
  the same order or f is (exactly) order g and
  write f??(g).
• Another, equivalent definition?(g) ? fR?R
  ?c1c2kgt0 ?xgtk c1g(x)?f(x)?c2g(x)
• - Everywhere beyond some point k, f(x) lies in
  between two multiples of g(x).

29
Rules for ?

• Mostly like rules for O( ), except
• ? f,ggt0 constants a,b?R, with bgt0, af ? ?(f),
  but ? Same as with O. f ? ?(fg)
  unless g?(1) ? Unlike O.f 1-b ? ?(f), and
  ? Unlike with O. (logb f)c ? ?(f).
  ? Unlike with O.
• The functions in the latter two cases we say are
  strictly of lower order than ?(f).

30
? Example

• Determine whether
• Quick solution

31
Other Order-of-Growth Relations

• ?(g) f g?O(f)The functions that are at
  least order g.
• o(g) f ?cgt0 ?k ?xgtk f(x) lt cg(x)The
  functions that are strictly lower order than g.
  o(g) ? O(g) ? ?(g).
• ?(g) f ?cgt0 ?k ?xgtk cg(x) lt f(x)The
  functions that are strictly higher order than g.
  ?(g) ? ?(g) ? ?(g).

32
Relations Between the Relations

• Subset relations between order-of-growth sets.

R?R
?( f )
O( f )
f
?( f )
?( f )
o( f )
33
Why o(f) ? O(x)??(x)

• A function that is O(x), but neither o(x) nor
  ?(x)

34
Strict Ordering of Functions

• Temporarily lets write f?g to mean f?o(g),
  fg to mean
  f??(g)
• Note that
• Let kgt1. Then the following are true1 ? log
  log n ? log n logk n ? logk n ? n1/k ? n ? n
  log n ? nk ? kn ? n! ? nn

35
Review Orders of Growth

• Definitions of order-of-growth sets, ?gR?R
• O(g) ? f ? cgt0 ?k ?xgtk f(x) lt cg(x)
• o(g) ? f ?cgt0 ?k ?xgtk f(x) lt cg(x)
• ?(g) ? f g?O(f)
• ?(g) ? f g?o(f)
• ?(g) ? O(g) ? ?(g)

36
(No Transcript)
37
Addition of Binary Numbers

• procedure add(an-1a0, bn-1b0 binary
  representations of non-negative integers a, b)
• carry 0
• for bitIndex 0 to n-1 go through bits
• bitSum abitIndexbbitIndexcarry 2-bit
  sum
• sbitIndex bitSum mod 2 low bit of
  sum
• carry ?bitSum / 2? high bit of sum
• sn carry
• return sns0 binary representation of integer s

38
Twos Complement

• In binary, negative numbers can be conveniently
  represented using twos complement notation.
• In this scheme, a string of n bits can represent
  any integer i such that -2n-1 i lt 2n-1.
• The bit in the highest-order bit-position (n-1)
  represents a coefficient multiplying -2n-1
• the other positions i lt n-1 just represent
  2i, as before.
• The negation of any n-bit twos complement number
  a an-1a0 is given by an-1a0 1.

The bitwise logical complement of the n-bit
string an-1a0.
39
Correctness of Negation Algorithm

• Theorem For an integer a represented in twos
  complement notation, -a a 1.
• Proof a -an-12n-1 an-22n-2 a020,
• so -a an-12n-1 - an-22n-2 - - a020.
• Note an-12n-1 (1-an-1)2n-1 2n-1 - an-12n-1.
• But 2n-1 2n-2 20 1.
• So we have -a - an-12n-1 (1-an-2)2n-2
  (1-a0)20 1 a 1.

40
Subtraction of Binary Numbers

• procedure subtract(an-1a0, bn-1b0 binary twos
  complement reps. of integers a, b)
• return add(a, add(b,1)) a (-b)
• Note that this fails if either of the adds causes
  a carry into or out of the n-1 position, since
  2n-22n-2 ? -2n-1, and -2n-1 (-2n-1) -2n
  isnt representable! We call this an overflow.

41
Multiplication of Binary Numbers

• procedure multiply(an-1a0, bn-1b0 binary
  representations of a,b?N)
• product 0
• for i 0 to n-1
• if bi 1 then
• product add(an-1a00i, product)
• return product

i extra 0-bitsappended afterthe digits of a
42
Binary Division with Remainder

• procedure div-mod(a,d ? Z) Quotient rem. of
  a/d.
• n max(length of a in bits, length of d in
  bits)
• for i n-1 downto 0
• if a d0i then Can we subtract at this
  position?
• qi 1 This bit of quotient is 1.
• a a - d0i Subtract to get remainder.
• else
• qi 0 This bit of quotient is 0.
• r a
• return q,r q quotient, r remainder