CSCI 1302

1
CSCI 1302

• Problem Complexity
• NP Complete

2
The Power of Exponents

• A rich king and a wise peasant

3
The Wise Peasants Pay

• Day(N) Pieces of Grain
• 1 2
• 2 4
• 3 8
• 4 16
• 63 9.223x1018
• 64 1.845x1019

2N
4
The Towers of Hanoi
A B
C

• Goal Move stack of rings to another peg
• Rule 1 May move only 1 ring at a time
• Rule 2 May never have larger ring on top of
  smaller ring

5
Towers of Hanoi Solution
Original State
Move 1
Move 2
Move 3
Move 5
Move 4
Move 6
Move 7
6
Towers of Hanoi - Complexity

• For 3 rings we have 7 operations.
• In general, the cost is 2N 1 O(2N)
• Each time we increment N, we double the amount of
  work.
• This grows incredibly fast!

7
The Bounded Tile Problem
Match up the patterns in thetiles. Can it be
done, yes or no?
8
Analysis of the Bounded Tiling Problem

• Tile a 5 by 5 area (N 25 tiles)
• 1st location 25 choices
• 2nd location 24 choices
• And so on
• Total number of arrangements
• 25 24 23 22 21 .... 3 2 1
• 25! (Factorial) 1.55x1025
• Bounded Tiling Problem is O(N!)

9
Performance Categories of Algorithms

• Sub-linear O(Log N)
• Linear O(N)
• Nearly linear O(N Log N)
• Quadratic O(N2)
• Exponential O(2N)
• O(N!)
• O(NN)

Polynomial
10
Reasonable vs. Unreasonable

• Reasonable algorithms have polynomial factors
• O (Log N)
• O (N)
• O (NK) where K is a constant
• Unreasonable algorithms have exponential factors
• O (2N)
• O (N!)
• O (NN)

11
Effects of Exponents
1050 1045 1040 1035 1030 1025 1020 1015 trillion b
illion million 1000 100 10
NN
2N
N5
N
Dont Care!
2 4 8 16 32 64 128 256 512 1024
N
12
Effects of Exponents

• Consider Towers of Hanoi (or other 2N algorithm)
  for N of only 256.
• Time cost is a number with 78 digits to the left
  of the decimal.
• For comparison
• Number of protons (estimated) in the known
  universe is a number with 77 digits.

13
What about a Faster Computer?

• What if
• Instructions took ZERO time to execute
• CPU registers could be loaded at the speed of
  light
• These algorithms are still unreasonable!
• The speed of light is only so fast!

14
Relations between Problems, Algorithms, and
Programs
Problem
Algorithm
. . . .
Algorithm
. . . .
Program
Program
Program
Program
. . . .
15
Cost and Complexity

• Algorithm complexity can be expressed in Order
  notation, e.g. at what rate does work grow with
  N?
• O(1) Constant
• O(logN) Sub-linear
• O(N) Linear
• O(NlogN) Nearly linear
• O(N2) Quadratic
• O(XN) Exponential
• But, for a given problem, how do we know if a
  better algorithm is possible?

16
The Problem of Sorting

• For example, in discussing the problem of
  sorting
• Two algorithms to solve
• Bubblesort O(N2)
• Mergesort O(N Log N)
• Can we do better than O(N Log N)?

17
Algorithm vs. Problem Complexity

• Algorithmic complexity is defined by analysis of
  an algorithm
• Problem complexity is defined by
• An upper bound defined by an algorithm
• A lower bound defined by a proof

18
The Upper Bound

• Defined by an algorithm
• Defines that we know we can do at least this good
• Perhaps we can do better
• Lowered by a better algorithm
• For problem X, the best algorithm was O(N3), but
  my new algorithm is O(N2).

19
The Lower Bound

• Defined by a proof
• Defines that we know we can do no better than
  this
• It may be worse
• Raised by a better proof
• For problem X, the strongest proof showed that
  it required O(N), but my new, stronger proof
  shows that it requires at least O(N2).

20
Upper and Lower Bounds

• The Upper bound is the best algorithmic solution
  that has been found for a problem.
• Whats the best that we know we can do?
• The Lower bound is the best solution that is
  theoretically possible.
• What cost can we prove is necessary?

21
Changing the Bounds
Lowered by betteralgorithm
Upper bound
Lower bound
Raised by betterproof
22
Open Problems

• The upper and lower bounds differ.

Lowered by betteralgorithm
Upper bound
Unknown
Lower bound
Raised by betterproof
23
Closed Problems

• The upper and lower bounds are identical.

Upper bound
Lower bound
24
Closed Problems

• Better algorithms are still possible
• Better algorithms will not provide an improvement
  detectable by Big O
• Better algorithms can improve the constant costs
  hidden in Big O characterizations

25
Tractable vs. Intractable

• Problems are tractable if the upper and lower
  bounds have only polynomial factors.
• O (log N)
• O (N)
• O (NK) where K is a constant
• Problems are intractable if the upper and lower
  bounds have an exponential factor.
• O (N!)
• O (NN)
• O (2N)

26
Terminology

• Polynomial algorithms are reasonable
• Polynomial problems are tractable
• Exponential algorithms are unreasonable
• Exponential problems are intractable

27
Terminology
28
Problems that Cross the Line

• The upper bound implies intractable
• The lower bound implies a tractable
• Could go either way

29
Problems that Cross the Line

• What if a problem has
• An exponential upper bound
• A polynomial lower bound
• We have only found exponential algorithms, so it
  appears to be intractable.
• But... we cant prove that an exponential
  solution is needed, we cant prove that a
  polynomial algorithm cannot be developed, so we
  cant say the problem is intractable...

30
NP-Complete Problems

• The upper bound suggests the problem is
  intractable
• The lower bound suggests the problem is tractable
• The lower bound is linear O(N)
• They are all reducible to each other
• If we find a reasonable algorithm (or prove
  intractability) for one, then we can do it for
  all of them!

31
Example NP-Complete Problems

• Path-Finding (Traveling salesman)
• Map coloring
• Scheduling and Matching (bin packing)
• 2-D arrangement problems
• Planning problems (pert planning)
• Clique

32
Traveling Salesman
33
5-Clique
34
Map Coloring
35
Class Scheduling Problem

• With N teachers with certain hour restrictions M
  classes to be scheduled, can we
• Schedule all the classes
• Make sure that no two teachers teach the same
  class at the same time
• No teacher is scheduled to teach two classes at
  once

36
Certificates

• Returning true in order to show that the
  schedule can be made, we only have to show one
  schedule that works
• This is called a certificate.
• Returning false in order to show that the
  schedule cannot be made, we must test all
  schedules.

37
Oracles

• If we could make the right decision at all
  decision points, then we can determine whether a
  solution is possible very quickly!
• If the found solution is valid, then True
• If the found solution is invalid, then False
• If we could find the certificates quickly,
  NP-complete problems would become tractable
  O(N)
• This (magic) process that can always make the
  right guess is called an Oracle.

38
Determinism vs. Nondeterminism

• Nondeterministic algorithms produce an answer by
  a series of correct guesses
• Deterministic algorithms (like those that a
  computer executes) make decisions based on
  information.

39
NP-Complete

• NP-Complete comes from
• Nondeterministic Polynomial
• Complete - Solve one, Solve them all
• There are more NP-Complete problems than provably
  intractable problems.

40
Become Famous!

• To get famous in a hurry, for any NP-Complete
  problem
• Raise the lower bound (via a stronger proof)
• Lower the upper bound (via a better algorithm)
• Theyll be naming buildings after you before you
  are dead!

41
FIN