P, NP, and NP-Complete

1
P, NP, and NP-Complete

• Suzan Köknar-Tezel

2
CSC 551Design and Analysis of Algorithms

• Giving credit where credit is due
• These lecture notes are based on slides by
• Dr. Cusack
• http//www.cse.unl.edu/goddard/Courses/CSCE310J/L
  ectures/Lecture10-NPcomplete.pdf
• Dr. Leubke
• http//www.cs.virginia.edu/luebke/cs332/index.htm
  l
• Dr. Goddard
• http//www.cse.unl.edu/goddard/Courses/CSCE310J
• And Wikipedia
• I have modified them and added new slides

3
Tractability

• Some problems are intractable
• As they grow large, we are unable to solve them
  in reasonable time
• What is reasonable time? Standard working
  definition polynomial time
• On input of size n, the worst-case running time
  is ?(nk)
• Polynomial time ?(n2), ?(n3), ?(1) ?(nlgn)
• Not in polynomial time ?(2n), ?(nn), ?(n!)

4
Polynomial-Time Algorithms

• Are some problems solvable in polynomial time?
• Of course every algorithm weve studied provides
  polynomial-time solution to some problem
• Are all problems solvable in polynomial-time?
• No
• Most are optimization problems

5
NP-completeness

• Near the end of 60s
• Growing list of problems with no efficient
  solution
• Remarkable discovery
• many of these problems were interrelated
• If one solved in polynomial time, all will be
  solved in polynomial time
• NP-completeness

6

• Our new focus showing a given problem cannot be
  solved efficiently!

7
Optimization/Decision Problems

• Optimization problems
• They ask What is the optimal solution to
  problem X?
• Examples
• 0-1 Knapsack
• Fractional Knapsack
• Minimum Spanning Tree
• Decision problems
• They ask Is there a solution to problem X with
  property Y?
• Example
• Does graph G have a MST of weight W?

8
Optimization/Decision Problems

• An optimization problem tries to find an optimal
  solution
• A decision problem tries to answer a yes/no
  question
• Many problems will have decision and optimization
  versions
• E.g MST
• Optimization find an MST of graph G
• Decision does graph G have a MST of weight lt W?

9

• Our new focus showing a given problem cannot be
  solved efficiently!
• We will phrase optimization problems as decision
  problems
• If the decision version (which is intuitively
  easier to solve) cannot be solved effciiently,
  the optimization version also cannot be solevd
  effciiently.

10
The Class P

• P the class of decision problems that have
  polynomial-time deterministic algorithms
• That is, they are solvable in ?(nk)
• A deterministic algorithm is (essentially) one
  that always computes the correct answer
• Sample problems in P
• Fractional knapsack
• MST
• Single-source shortest path
• Sorting

11
The Class NP

• NP (nondeterministic polynomial-time) the class
  of decision problems that are verifiable in
  polynomial time.

12
Sample Problems in NP

• Fractional Knapsack
• MST
• Single-source shortest path
• Sorting
• Others?
• Knapsack
• Hamiltonian Cycle Problem
• Satisfiability (SAT)
• Conjunctive Normal Form (CNF) SAT
• 3-CNF SAT

13
Hamiltonian Cycle Problem (HCP)

• A hamiltonian-cycle of an undirected graph is a
  simple cycle that contains every vertex exactly
  once and returns to the starting vertex
• The Hamiltonian-Cycle Problem given an
  undirected graph G, does it have a hamiltonian
  cycle?
• The HCP is in NP
• A solution is easy to verify in polynomial time
  (how?)

13
14
The MILLION DOLLAR Question

• Does P NP?
• This is literally a million dollar question
• The Millennium Problems
• The Clay Mathematics Institute of Cambridge,
  Massachusetts (CMI) has designated 7 problems as
  Millennium Problems
• http//www.claymath.org/millennium/
• Each problem has a prize of 1,000,000.00 to
  whomever solves it
• P NP is one of the seven problems

15
P and NP

• What do we mean when we say a problem is in P?
• A A solution can be found in polynomial time
• What do we mean when we say a problem is in NP?
• A A solution can be verified in polynomial time
• What is the relation between P and NP?
• A P ? NP, but no one knows whether P NP

15
16
NP-Complete Problems

• NPC problems are the hardest problems in NP
• If any one NPC problem can be solved in
  polynomial time
• then every NPC problem can be solved in
  polynomial time
• and in fact every problem in NP can be solved
  in polynomial time (which would show that P NP)
• Thus solve the hamiltonian-cycle in ?(n100)
  time, youve proved that P NP. Retire rich
  famous.

17
Reduction

• The crux of NP-Completeness is reducibility
• Informally, a problem P can be reduced to another
  problem Q if any instance of P can be easily
  rephrased as an instance of Q, the solution to
  which provides a solution to the instance of P
• What do you suppose easily means?
• This rephrasing is called transformation
• Intuitively If P reduces to Q easily, P is no
  harder to solve than Q

18
Reducibility

• An example
• P Given a set of Booleans, is at least one TRUE?
• Q Given a set of integers, is their sum
  positive?
• Transformation (x1, x2, , xn) (y1, y2, ,
  yn) where yi 1 if xi TRUE, yi 0 if xi
  FALSE

19
Using Reductions

• If P is polynomial-time reducible to Q, we denote
  this P P Q
• Definition of NP-Complete A decision problem Q
  is NPC iff
• Q ? NP (What does this mean?)
• Every problem in NP is polynomial-time reducible
  to Q
• I.e. R P Q for every R ? NP

20
NP-Hard Problems

• Definition P is NP-Hard iff
• Every problem in NP is polynomial-time reducible
  to P
• I.e. R P P for every R ? NP
• So, an alternative definition for NPC
• P is NPC iff P is NP-Hard and P ? NP
• Important For a problem to be NP-hard it does
  not have to be in class NP

20
21
The Relationship between P, NP, and NP-Hard

• If there is a polynomial algorithm for any
  NP-hard problem
• Then there are polynomial algorithms for all
  problems in NP
• And hence P NP
• If P ? NP, then NP-hard problems have no
  solutions in polynomial time
• If P NP, it does not resolve whether the
  NP-hard problems can be solved in polynomial time

22
The Relationship between P, NP, and NP-Hard

• possibilities

If P ? NP
If P NP
23
Proving NP-Completeness

• Steps to prove a problem Q is NPC?
• Prove Q ? NP
• Pick a known NPC problem P
• Reduce P to Q
• Describe a transformation that maps instances of
  P to instances of Q, such that yes for Q
  yes for P
• Prove the transformation works
• Prove it runs in polynomial time

NP-hard
24
Why reduction to single NPC problem P is okay?

• If P is NPC, then every problem in NP is
  polynomial-time reducible to P
• Transitivity If P can be polynomial reducible to
  Q, then every problem in NP is polynomial-time
  reducible to Q
• Hence, If P P Q and P is NPC, Q is also NPC
• This is the key idea for today

25
Why Prove NP-Completeness?

• Though nobody has proven that P ? NP, if you
  prove a problem NP-Complete, most people accept
  that it is probably intractable
• Therefore it can be important to prove that a
  problem is NP-Complete
• Dont need to come up with an efficient algorithm
• Can instead work on approximation algorithms

26
NPC Problems

• We will now look at some NPC problems
• In some cases we will also look at the
  transformations

27
Hamiltonian Cycle Problem

• A hamiltonian-cycle of an undirected graph is a
  simple cycle that contains every vertex exactly
  once and returns to the starting vertex
• The Hamiltonian-Cycle Problem given an
  undirected graph G, does it have a hamiltonian
  cycle?
• The HCP is in NPC

28
Traveling salesman problem (TSP)

• The well-known traveling salesman problem
• Optimization variant a salesman must travel to n
  cities, visiting each city exactly once and
  finishing where he begins. How to minimize
  travel time?
• Model as complete graph with cost c(i,j) to go
  from city i to city j
• How would we turn this into a decision problem?
• A ask if ? a TSP with cost lt k

29
Hamiltonian Cycle ? TSP

• The steps to prove TSP is NP-Complete
• Prove that TSP ? NP (Argue this)
• Reduce the undirected hamiltonian cycle problem
  to the TSP
• So if we had a TSP-solver, we could use it to
  solve the hamilitonian cycle problem in
  polynomial time
• How can we transform an instance of the
  hamiltonian cycle problem to an instance of the
  TSP?
• Can we do this in polynomial time?

30
Transformation Hamiltonian Cycle ? TSP

• Let G (V,E) be a graph with n nodes
• Ham. cycle problem
• Construct an instance of TSP as follows
• Let G (V,VxV) be a complete graph on the
  vertices of G
• The edge weights c(u,v) are 1 if the edge (u,v)
  is in E and 2 otherwise
• The bound k is n where n is the number of
  vertices in V

31
Transformation Hamiltonian Cycle ? TSP
No Ham. Cycle
1
u
v
u
v
1
1
1
2
w
x
w
x
2
The best tour has c of 5, which is greater than
n, so NO ham. cycle
32
Transformation Hamiltonian Cycle ? TSP
Ham. Cycle Present
1
u
v
u
v
1
1
1
1
w
x
w
x
2
The best tour has c of 4, which is equal to n, so
? ham. cycle
33
Transformation Hamiltonian Cycle ? TSP

• A hamiltonian cycle in G is a tour in G with
  cost n.
• If there are no hamiltonian cycles in G, any tour
  in G must cost at least n1
• So G has a hamiltonian cycle iff G has a
  traveling salesperson tour of weight n
• Is this a polynomial-time transformation?

34
The Satisfiability (SAT) Problem

• Satisfiability (SAT)
• Given a boolean expression on n variables, can we
  assign values such that the expression is TRUE?
• Ex ((x1?x2) ? ?((?x1?x3) ? x4)) ? ?x2
• Simple enough but no known deterministic
  polynomial time algorithm exists
• Easy to verify in polynomial time
• This is the first NPC problem
• Proven by Stephen Cook in 1971

35
Conjunctive Normal Form (CNF)

• Even if the form of the boolean expression is
  simplified, no known polynomial time algorithm
  exists
• Literal An occurrence of a boolean or its
  negation
• A boolean formula is in CNF if it is an AND of
  clauses, each of which is an OR of literals
• Ex (x1 ? ? x2) ? (?x1 ? x3 ? x4) ? (?x5 )
• 3-CNF Each clause has exactly 3 distinct
  literals
• Ex (x1 ? ?x2 ? ?x3 ) ? (?x1 ? x3 ? x4) ? (?x5 ?
  x3 ? x4 )
• Notice TRUE if at least one literal in each
  clause is true

36
3SAT Problem

• Satisfiability of Boolean formulas in 3-CNF form
  (the 3SAT Problem) is NP-Complete
• Proof Nope
• The reason we care about the 3-CNF problem is
  that it is relatively easy to reduce to others
• Thus by proving 3SAT NP-Complete we can prove
  many seemingly unrelated problems NP-Complete

37
3SAT ? Clique

• What is a clique of a graph G?
• A a subset of vertices fully connected to each
  other, i.e. a complete subgraph of G
• The clique problem how large is the maximum-size
  clique in a graph?
• Can we turn this into a decision problem?
• A Yes, we call this the k-clique problem
• Is the k-clique problem within NP?

38
3SAT ? Clique

• What should the reduction do?
• A Transform a 3-CNF formula to a graph, for
  which a k-clique will exist (for some k) iff the
  3-CNF formula is satisfiable

39
Transformation 3SAT ? Clique

• The reduction
• Let B C1 ? C2 ? ? Ck be a 3-CNF formula with
  k clauses, each of which has 3 distinct literals
• For each clause put a triple of vertices in the
  graph, one for each literal
• Put an edge between two vertices if they are in
  different triples and their literals are
  consistent, meaning not each others negation

40
Transformation 3SAT ? Clique
B (x ? ?y ? ?z) ? (?x ? y ? z ) ? (x ? y ? z )
?y
x
?z
?x
x
y
y
z
z
41
Transformation 3SAT ? Clique

• Prove the reduction works
• If B has a satisfying assignment, then each
  clause has at least one literal (vertex) that
  evaluates to 1
• Picking one such true literal from each clause
  gives a set V of k vertices. V is a clique
  (Why?)
• If G has a clique V of size k, it must contain
  one vertex in each triple (clause) (Why?)
• We can assign 1 to each literal corresponding
  with a vertex in V, without fear of contradiction

42
Transformation 3SAT ? Clique
B (x ? ?y ? ?z) ? (?x ? y ? z ) ? (x ? y ? z )
?y
x
?z
Assume the satisfying assignment is x 1 y
1 z 0
So we know we have chosen k vertices, and those k
vertices must be a clique
?x
x
y
y
z
z
43
Transformation 3SAT ? Clique
B (x ? ?y ? ?z) ? (?x ? y ? z ) ? (x ? y ? z )
?y
x
?z
Make the assignment x 1 ?y 1 z 1 and we
know that each triple MUST contain one of
these and thus B is satisfied
Assume there is a clique of size 3
?x
x
y
y
z
z
44
General Comments

• Literally hundreds of problems have been shown to
  be NP-Complete
• Some reductions are profound, some are
  comparatively easy, many are easy once the key
  insight is given

45
Other NP-Complete Problems

• Subset-sum Given a set of integers, does there
  exist a subset that adds up to some target T?
• 0-1 knapsack when weights not just integers
• Hamiltonian path Obvious
• Graph coloring can a given graph be colored with
  k colors such that no adjacent vertices are the
  same color?
• Etc

46
Review P and NP

• What do we mean when we say a problem is in P?
• A A solution can be found in polynomial time
• What do we mean when we say a problem is in NP?
• A A solution can be verified in polynomial time
• What is the relation between P and NP?
• A P ? NP, but no one knows whether P NP

47
Review NP-Complete

• What, intuitively, does it mean if we can reduce
  problem P to problem Q?
• P is no harder than Q
• How do we reduce P to Q?
• Transform instances of P to instances of Q in
  polynomial time s.t. Q yes iff P yes
• What does it mean if Q is NP-Hard?
• Every problem P?NP ?p Q
• What does it mean if Q is NP-Complete?
• Q is NP-Hard and Q ? NP

48
Review Proving Problems NP-Complete

• How do we usually prove that a problem R is
  NP-Complete?
• A Show R ?NP, and reduce a known NP-Complete
  problem Q to R