This lecture concludes the examinable material

1
This lecture concludes the examinable material!

• Plan
• 12 December Encryption/begin review
• 5 January No tutorials! (but study!)
• 9 January Mock exam, be prepared
• 12 January Discuss mock exam
• 16 January Final review session
• 2-4 PM, 4 Feb. Provisional exam time
• 10 December Geometry of the Universe
• 6 PM Portland David Wands

2
11. P, NP and NP - Complete

• So far we have focussed on two kinds of problems
• Solvable and unsolvable problems
• Solvable problems can be split into three
  categories
• Tractable (possible and practically solvable)
• Intractable (possible but impractical)
• Those which seem to be intractable, but which
  may possibly be tractable

3
What do we mean by intractable?

• A problem is said to be intractable if it is
  impossible to solve with a polynomial-time
  algorithm
• Definition
• A polynomial-time algorithm is one whose
  worst-case time complexity is bounded above by a
  polynomial function of its input size.
• That is, if n is the input size, there exists a
  polynomial p(n) such that
• W(n) ? O(p(n))

4

• Examples of various kinds of problems
• I Tractable (polynomial-time algorithms found)
• sorting
• searching a sorted array
• matrix multiplication
• shortest path
• optimal binary search tree problem
• II Apparently intractable (but not proven so)
• Travelling salesman problem
• Knapsack problem

5

• Known to be intractable
• III Proven intractable (super-polynomial)
• Fibonacci sequence
• Tower of Hanoi
• IV Undecidable (no algorithm can ever exist)
• Halting Problem
• Problems equivalent to the halting problem
• Post correspondence problem
• Equivalence problem
• Hilberts tenth problem
• Tiling problem

6
Tower of Hanoi
Move the rings one at a time, transferring the
whole stack to another post, never placing a ring
on one smaller than itself
A simple recursive solution is known, but it
takes 2n 1 steps, so is solvable but intractable
7
Knapsack Travelling salesman
8

• Decision problems
• A decision problem (also called a test) is one
  whose outputs are just yes or no.
• Most problems can be written in decision form,
  which is useful to define complexity classes
• Travelling salesperson decision problem
• Given n cities, the pairwise distance between
  them and a number K gt 0, is it possible to
  complete a round trip in less than K miles?

9
 

• Complexity classes
• Decision problems can be separated into various
  complexity classes
• The simplest of these is P
• Definition
• P is the set of all decision problems that can be
  solved by polynomial-time algorithms

10

• What kinds of problems are in P?
• Determining if a key is in an array,
• or if a key is in a sorted array
• 2. Decision problems corresponding to
  optimization problems for which we found a
  polynomial time algorithm.
• What are not in P?
• The proven intractable problems are the only ones
  we know for sure.

11

• A related complexity class NP
• This class contains problems whose solutions can
  be verified in polynomial time
• Consider the Travelling Salesperson problem
• If someone claimed that they had a solution to
  some instance, then we can check this quickly
• That is, for a particular graph and a particular
  number d, we can test if a particular tour
  takes a distance less than or equal to d.
• An algorithm to do this would look like

12

• function verify(Gweighted_digraph
• d number
• Sclaimed_tour) boolean
• begin
• if S is a tour and the total weight of the edges
  in S is d then
• verify true
• else
• verify false
• end
• end

13

• Nondeterministic algorithm
• Think of the solution as two stages
• 1. Guessing Stage (nondeterministic part)
• Make a guess at a solution, either out of
  thin air or by waving a magic wand
• 2. Verification Stage
• deterministic algorithm which checks if the
  guess is a solution, definitely halting in every
  true case .

14
Non-deterministic decision problems

•  A non-deterministic algorithm "solves" a
  decision problem if
• 1. For any instance for which the answer is
  "yes", there is some string S for which the
• verification stage returns "true".
•  
• 2. For any instance for which the answer is "no"
  there is no string for which the verification
  stage returns "true".

15

• The complexity class NP
• A polynomial-time nondeterministic algorithm is a
  nondeterministic algorithm whose verification
  stage is a polynomial-time algorithm
• Definition
• NP is the set of all decision problems that can
  be solved by polynomial-time non-deterministic
  algorithms

16

• Everything in P is in NP!
• Equivalently,
• P ? NP,
• since if it is possible to construct a
  polynomial solution then we can use the same
  algorithm to check any proposed solution.
• Does NP P ?
• We don't know, but theres a 1,000,000 prize if
  you can find the answer!
• No one has found any problems in NP which are
  definitely not in P, or proved that all problems
  in NP are also in P

17

• NP-complete the hardest problems in NP
• In 1971, Cook discovered that a number of open
  problems were among the hardest in NP
• If a polynomial time algorithm were ever found
  for one of these problems, then there would be a
  polynomial time algorithm for every problem in
  NP!
• Logical Truth (CNF satisfiability problem)
• Given a statement in logic (in conjunctive normal
  form) can we assign truth values to the
  variables so that the entire sentence turns out
  true?

18

• Any problem that is one of these hardest problems
  is said to be NP - complete.
• NP-complete problems are all effectively
  equivalent in the sense that if any one of them
  is in P, then they all are.
• If any NP-complete problem is ever shown to have
  a polynomial algorithm we could deduce that P
  NP.
• In fact it is conjectured that P ¹ NP.

19

• NP-complete examples
• Logical (CNF) satisfiability
• Travelling salesperson
• Knapsack problem (0/1 version)
• Hamiltonian path problem
• Map coloring problem
• Timetabling problem
• They all admit unreasonable, exponential-time
  solutions, but none of them are known to admit
  reasonable ones.
• Moreover no-one has been able to prove that any
  of them require super-polynomial time

20

• NP-complete examples
•  
• 0/1 knapsack decision problem
• Given a collection of objects, each of which has
  a value and a weight, can a value of at least V
  be achieved without exceeding a total weight W?
• From trying to choose treasures to stuff into
  your knapsack, assuming you can only carry so
  much weight.

21
NP-complete examples

• Bin-packing problem
• Given n crates of different weights, are t
  trucks, each of which can carry a load w,
  sufficient to transport the weights?

22

• NP-complete examples
•  
• Hamiltonian paths
• Given a graph consisting of points and edges, is
  there a path along the edges such that every node
  gets visited exactly once, except the first node
  of the path which is equal to the last one.
• (Notice that if we want a path to pass through
  all edges exactly once, a Euler path,it is a
  different story.)

23

• Which of the graphs below has a Hamiltonian path?

24

• NP-complete examples
• Colouring Maps and Graphs
• Given a map can we colour it using only three
  colours? We must avoid having adjoining areas of
  the same colour.
• Given a graph what is the least number of
  colours we need to colour it so that no two
  neighbouring nodes can be the same colour?

25

• NP-complete examples
•  
• Timetabling decision problem
• Given a list of subjects and students enrolled
  in them, as well as the number of time slots
  available, is it possible to timetable the
  subjects so that no student has a clash?
• Other examples
• Some well known games fall into this category,
  including Tetris, Minesweeper and the 15 puzzle.

26
How are NP-complete problems equivalent?

• Every NP-complete problem is polynomially
  reducible to every other.
• Given two problems, a polynomial-time reduction
  is an algorithm that runs in polynomial time and
  reduces one problem to the other.
• If we have an input X to the first problem and
  want a yes or no answer, we can transform X
  into an input Y for the second problem in such a
  way that the seconds problem answer for Y is
  precisely the first problems answer for X.

27

• Is your problem NP-complete?
• To show that a new problem A is NP-complete, all
  we need do is establish a polynomial reduction of
  A to a problem already known to be NP-complete
  and then to reduce another such problem to A.
• The first reduction shows that A cannot be any
  harder than NP-complete and the second shows it
  cannot be any easier than NP-complete.

28

• Examples of reduction
• The Hamiltonian circuit problem (HCP) can be
  reduced to the Travelling salesperson problem
  (TSP).
• For any given path diagram, one can find an
  equivalent map of cities for the travelling
  salesperson problem
• Basic idea associate each node with one city,
  and if a path exists between two nodes, then
  assign a very short distance between the
  corresponding cities. (See notes for details.)

29
Logical satisfiability

• The first NP-complete problem discovered by Cook
  was general logical satisfiability
• Given a logical expression in conjunctive normal
  form, can we find truth values for the
  variables which make it true?
• Recall conjunctive normal form (CNF) is a number
  of fundamental disjunctions (ors) connected by
  conjunctions (ands)
• For example,
• (x ? ?y ? z) ? (x ? ?y) ? (?x ? y ? w ? v)

30
2 and 3 Satisfiability problems

• We can consider simpler versions of the general
  CNF satisfiability problem by restricting the
  number of literals in each fundamental
  disjunction.
• 2-satisfiability
• At most two literals in each disjunction. E.g.,
• (w ? ?y) ? (x ? ?y) ? (z)
• 3-satisfiability
• At most three literals in each disjunction.
  E.g.,
• (x ? ?y ? z) ? (x ? ?y) ? (?x ? y ? w)

31
How hard are the restricted problems?

• Cook showed that the general CNF problem could be
  polynomially reduced to the
  3-satisfiability problem
• Thus, the 3-satisfiability problem is also
  NP-complete!
• However, one can show that the 2-satisfiability
  problem is much simpler, and can in fact be
  solved in polynomial time
• Thus, the 2-satisfiability problem is in P !
• An algorithm to do this is described in the
  notes.

32
Where do we stand?
But if P NP, P would encompass both the NP and
NP-complete regions
33

• What lies between P and NP?
• If P ? NP is true, could there be problems in
  NP - P but not NP-complete?
• We conjecture that the primeness problem and the
  compositeness problem are examples of such.
• Why?
• Because there are polynomial algorithms that
  almost always correctly solve these problems.

34
The state of NP

• P? NP ? P? NP-complete Æ
• We suspect that if P ? NP, then the set
• NP - (P È NP - complete) is not empty.
• One problem that may not be NP-complete is
• Factorisation Problem
• Given a positive integer n, are there integers
  mgt1 and kgt1 such that n mk?

35
NP -hard

• A problem is NP-hard if all NP problems can be
  polynomially reduced to it.
• So the difference between NP-complete and NP-hard
  is that an NP-complete problem must be in NP.
• An NP-hard problem need not be in NP.

36
Why is it helpful to study P, NP, etc.?

• First, it is generally useful to study what
  problems have been considered before, so we can
  recognise them when we encounter them and not try
  to reinvent a new solution.
• If we can identify that our problem is
  NP-complete, we know its probably not worth
  looking for a perfect solution.
• We can then spend our time looking for a solution
  which is not ideal, but which fits our needs.

37
Other Methods if you cant find an efficient
solution

• 1. Find a probabilistic polynomial algorithm.
• This is an algorithm which is correct in all but
  a very small number of cases.
• Very few problems admit such an algorithm.
• One that does is the primeness problem. Details
  are in Harel (1987).

38

• It is based on the selection of k random numbers
  between 1 and a-1, where a is the n-digit number
  to be tested for primeness.
• If a is prime the algorithm always answers yes.
  
• If a is not prime, then there is a small
  probability that the algorithm will answer yes
  instead of no.
• This probability is less than 1/ 2 k .

39

• By choosing a reasonably large number k of random
  numbers, like 100, this probability is less than
  the probability of hardware failing.
• Because of this we think of the primeness and
  compositeness problems as being effectively
  feasible.
• Unfortunately (or actually fortunately) answering
  the question Is N a prime number? is much
  easier than finding the factors of N

40

• 2. Restrict the problem somehow to suit our needs
  and then see if this is in P.
• E.g. consider the bin packing problem.
• If we make the capacity w of each truck the same,
  the problem is solvable in polynomial time by
  exhaustive search.
• For the timetabling problem there are polynomial
  solutions in the case when there are less than
  three subjects.

41

• 3. Find an approximate solution which is in P.
• Two types exist.
• We can find solutions which although not optimal
  will not be too far away in every case.
• Or we can find solutions which in rare cases are
  wildly wrong but otherwise are very close to
  optimal.
• For example, the travelling salesman problem
  there are polynomial algorithms which compute a
  route that is never more than 60 longer than the
  optimal route.

42
A final unsolvable problem The Tiling problem

• Given a finite set T of tile types, can any
  finite area, of any size, be covered using only
  tiles in T such that the colours on any two
  touching edges are the same?
• Tiles have a fixed orientation and cannot be
  rotated.

43
T
44
(No Transcript)
45
(No Transcript)
46
(No Transcript)
47
(No Transcript)
48
(No Transcript)
49
(No Transcript)
50
(No Transcript)
51
(No Transcript)
52
(No Transcript)
53
Another tiling example
T
54
(No Transcript)
55
(No Transcript)
56
(No Transcript)
57
(No Transcript)
58
(No Transcript)
59
(No Transcript)
60
(No Transcript)
61
? ?
62
The end of the exam course-work!

• Next year we will review and study some
  interesting implications/applications
• Ask yourself
• What do you think the main results of this course
  are?
• Why is this course compulsory? If you think it is
  useless maybe you are missing something!
• NOTE no tutorial tomorrow please come today at
  1pm, 4pm or 5pm.