Reductions

1
Reductions NP-Completeness
Thinking about Algorithms Abstractly

• Jeff Edmonds
• York University

COSC 3101
Lecture 9
2
Reductions NP-Completeness

• Cryptography
• Halting Problem
• Goethe's Incompleteness
• Halting Math Truth

• History
• Course vs Schdule
• NP
• Reductions
• Give algorithm for matching
• Prove problems are hard
• Clique vs Independent Set
• 12 Step Program
• Any SAT
• Trivial Reductions
• SAT 3-COL
• Summary Reductions

There is a custom slide show called 3101 that
deletes someof these slides.
3
An early version of these slides produced by

• Steven Rudich
• from Carnegie Mellon University

4
Graph Colouring
Colour each node. Nodes with lines between them
must have different colours. Can you do it with 2
colours? Can you do it with 3?
5
Which are more alike?
Circuit Satisfiability
Graph Colouring
BipartiteMatching
Network Flow
Similar structure
Similar structure
Non-Deterministic Poly Time Complete
6
Which are more alike?
7
History of Classifying Problems
In the beginning
And it was good.
He gave a new understanding of Algorithm
8
History of Classifying Problems
As one gets closer to God, everything is
possible. We must only learn how to do it.
Euclid (300 BC)
GCD
9
History of Classifying Problems
As one gets closer to God, everything is
possible. We must only learn how to do it.
Is every thing possible?
Or are some problemsUncomputable?
Need to formally define Algorithm
GCD
10
History of Classifying Problems
Need to formally define Algorithm
GCD
11
History of Classifying Problems
Halting
Halting Problem Does a given algorithm Halt?
GCD
12
History of Classifying Problems
Halting
GCD
13
History of Classifying Problems
Halting
Matching
Easy to prove
Matching
Matching Problem Does a graph have a matching?
Hard to prove
GCD
14
History of Classifying Problems
Halting
Non-Deterministic Polynomial Time

• There are lots of important search problems
• exponential time to search
• poly time to verify given witness

Matching is an example and it is in poly time.
Matching
Matching Problem Does a graph have a matching?
GCD
15
History of Classifying Problems
Halting
Non-Deterministic Polynomial Time

• There are lots of important search problems
• exponential time to search
• poly time to verify given witness

SAT
Matching
Circuit-Sat Problem Does a circuit have a
satisfying assignment.
GCD
16
History of Classifying Problems
Halting
Games
Are these easier than other exponential problems?
SAT
Matching
Circuit-Sat Problem Does a circuit have a
satisfying assignment.
GCD
17
History of Classifying Problems
Halting
Games
Are these harder than poly-time problems?
SAT
Matching
Circuit-Sat Problem Does a circuit have a
satisfying assignment.
GCD
18
History of Classifying Problems
Halting
Games
SAT
?poly Prob
SAT
Lets compare the problems in NP
None are harder than SAT!
Matching
Circuit-Sat Problem Does a circuit have a
satisfying assignment.
GCD
19
History of Classifying Problems
Halting
Games
SAT
Definition is asymmetric. There is a witness
that a circuit has a satisfying
assignment. There is no known witness that a
circuit has no satisfying assignment.
Matching
GCD
20
History of Classifying Problems
Halting
Games
SAT
Primality
Poly
Primality It an integer a prime. There is a
witness that it is not a prime. There is a
witness that is a prime. Recently, a poly time
alg was found.
Matching
GCD
21
History of Classifying Problems
Halting
Games
SAT
Poly

• Most natural problems in NP are either
• in PolyTime
• or NP-Complete
• except?

Matching
GCD
22
History of Classifying Problems
Halting
Games
SAT
Factoring Graph Isomorphism
Poly

• Most natural problems in NP are either
• in PolyTime
• or NP-Complete
• except

Matching
GCD
23
History of Classifying Problems
Halting
Games
SAT
Poly
There are surely problemsin poly-time for which
we dont yet have a known algorithm. Your job is
to find a algorithm for them.
Matching
GCD
24
History of Classifying Problems
Halting
There are more classes of problemsthat we wont
cover.
Games
SAT
Poly
Matching
GCD
25
History of Classifying Problems
Halting
These lectures is about understanding
NP-Completeness
Games
SAT
Poly
Matching
GCD
26
Course Scheduling
Given courses students want and the time slots
available schedule courses to time slotsto
minimize number of conflicts, i.e. student wants
two course that are scheduled at the same time.
27
K-CRAYOLA Problem
Colour each node. Nodes with lines between them
must have different colours.

• Given a graph G and a k, find a way to coloured
  G with k colours.

• Rudich www.discretemath.com

28
Two Different Problems
Schedule each course.Courses that conflict cant
be at same time.
Colour each node.Nodes with lines between them
must have different colours.
29
Problems are the Same!
Schedule each course.Courses that conflict cant
be at same time.
Colour each node.Nodes with lines between them
must have different colours.

Instance
Solution
30
K-CLIQUES

• A K-clique is a set of K nodes with all k(K-1)/2
  possible edges between them.

31
This graph contains a 4-clique
32
Given an n-node graph G and a number k, does G
contains a k-clique?
Brute Force Try out all n choose k possible
locations for clique.
Exponential number? if k ?(n) then 2?(n)
subsets to check if k3 than
only O(n3)
33
Given an n-node graph G and a number k, does G
contains a k-clique?
Inspiration? Whether there is a faster algorithm
is one of the biggest open problems in the
mathematics computer sci.
34
INDEPENDENT SET

• An independent set is a set of node with no edges
  between them.

35
Given an n-node graph G and a number k, does G
contains an independent set of size k?
Brute Force Exponential time. Inspiration
????
36

• These problems have a common story
• A large space of possible solutions
• Only a tiny fraction of which satisfy the
  constraints.
• Brute force takes too long,
• No feasible algorithm is known.
• Our goal is to study how they are
• cosmetically different,
• but fundamentally the same.

• Rudich www.discretemath.com

37
Non-Deterministic Poly-Time (NP)
Non-Deterministic Polynomial Time
Clique

• There are lots of important search problems
• exponential time to search
• poly time to verify given witness

Clique Problem Does a graph have a clique.
GCD
38
Optimization Problems

• Ingredients
• Instances The possible inputs to the problem.
• Solutions for Instance Each instance has an
  exponentially large set of solutions.
• Cost of Solution Each solution has an easy to
  compute cost or value.

39
Optimization Problems

• Specification of an Optimization Problem
• Preconditions The input is one instance.
• Postconditions The output is one of the valid
  solutions for this instance with optimal cost.
  (minimum or maximum)

Eg Given graph G, find biggest clique.
40
Non-DeterministicPoly-Time Decision Problems (NP)

• An optimization problem
• Each solution is either valid or not (no cost)
• The output is
• Yes, it has an valid solution.
• No, it does not
• the solution not returned
• Eg Given graph and integer ltG,kgt,
• does G have a clique of size k?

41
Non-DeterministicPoly-Time Decision Problems (NP)

• Key Given
• an instance I ( ltG,kgt)
• and a solution S ( subset of nodes)
• there is a poly-time alg Valid(I,S) to test
  whether or not S is a valid solution for I.
• Poly-time in I not in S. S cant be too big.
  

Not Valid
Valid
Formal definition Prob ? NP iff ? poly time
Valid such that Prob(I) ? S Valid(I,S)
42
Non-DeterministicPoly-Time Decision Problems (NP)

• Key
• If the instance has a valid solution
• A non-deterministic (fairy god mother) could
  prove it to you by giving you
• such a solution as a witness.
• You could check that it is valid.
• You could convince your boss.

Valid
43
Non-DeterministicPoly-Time Decision Problems (NP)

• Key
• If the instance does not have a valid solution
• A non-deterministic (fairy god mother) could
  prove it to you by giving you
• ????
• You have no way to convince your boss.

k5
44
Non-Deterministic Poly-Time (NP)

• Example 3-Col
• Instance A graph G.
• Solution Colouring C nodes of G with 3 colours
  such that every edge has two
  colours.
• G is a Yes instance if there is such a colouring.
  
• Given an instance G and a solution C, there is
  a poly-time alg Valid(G,C) to test whether or
  not C is a valid 3-colouring of G.
• ? 3-Col ? NP.

45
Non-Deterministic Poly-Time (NP)

• Example Not 3-Col
• Instance A graph G.
• Solution Colouring C nodes of G with 3 colours
  such that every edge has two
  colours.
• G is a No instance if there is such a colouring.
• Given an instance G and a solution C, there is
  a poly-time alg Valid(G,C) to test whether or
  not C is a valid 3-colouring of G.
• ? Not 3-Col ? NP.
• ? Not 3-Col ? Co-NP.

I cant witnessYes instances
46
Non-Deterministic Poly-Time (NP)
3-Col
Definition is asymmetric. There is a witness
that a graph has a 3-Col. There is no known
witness that a graph has no 3-Col.
GCD
47
Non-Deterministic Poly-Time (NP)

• Example Sorting
• Instance A list of numbers Xx1,xn.
• Solution Sorted order X.
• Given an instance X and a solution X, there is
  a poly-time alg Valid(X,X) to test whether or
  not X is a sorting of X.
• ? Sorting ? NP.
• Sorting is not a decision (Yes/No) problem.

X5,3,7,2
X2,3,5,7
48
Non-Deterministic Poly-Time (NP)

• Example Parity
• Instance A list of bits Xx1,xn.
• X is a Yes Instance if the number of xi1 is odd.
• ? Parity ? Poly-Time ? NP.

X0,1,0,11,0
Dont be lazy.You can solve this on your own.
What do I give as a witness?
parity1
49
Non-Deterministic Poly-Time (NP)
Games
3-Col
Parity ? Poly-Time ? NP.
Parity
50
Non-Deterministic Poly-Time (NP)

• Example Airplane Wing
• Instance Requirements I of the wing.
• Solution A description S of how to make the
  wing.
• I is a Yes instance if there is such a wing.
• Given an instance I and a proof S, there is a
  poly-time alg Valid(I,S) to test whether or
  not S is a valid solution for I.
• Airplane Wing ? NP.

I weight, lift, cost,
51
Non-Deterministic Poly-Time (NP)

• Example Math Truth
• Instance A math statement I
• Solution A proof S.
• I is a Yes instance if there is such a proof.
• Given an instance I and a proof S, there is a
  poly-time alg Valid(I,S) to test whether or
  not S is a valid solution for I.
• Math Truth ? NP.
• Witness is way too big!

in I not in S
I ? a,b,c,r ? 3 arbr ?cr
52
Non-Deterministic Poly-Time (NP)

• Example Math Truth
• Instance A math statement I
• Solution A proof S.
• I is a Yes instance if there is such a proof.
• Math Statement I can encode whether a TM stops
• Halting problem ?poly Math Truth
• ? Math Truth ? Undecidable.

I ? a,b,c,r ? 3 arbr ?cr
53
Non-Deterministic Poly-Time (NP)
Math Truth
Halting
Games
3-Col
GCD
54
Which are more alike?
Circuit Satisfiability
Graph Colouring
BipartiteMatching
Network Flow
Similar structure
Similar structure
Non-Deterministic Poly Time Complete
55
Two Different Problems
Schedule each course.Courses that conflict cant
be at same time.
Colour each node.Nodes with lines between them
must have different colours.

Two problems that are cosmetically different, but
substantially the same
56
Algorithms Inter-Changeable

Proof by Reductions
57
Reductions

• Reduction Design a fast algorithm for one
  computational problem, using a supposedly fast
  algorithm for another problem as a subroutine.
• Use to compare the two problems.
• Even if we dont know whether they can be solved
  in polynomial time or not,
• we can learn that either they both can or neither
  can.
• We can also learn that they have a similar
  structure.

58
Reductions
Design a fast algorithm for Palg using a
supposed fast algorithm for Poracle as a
subroutine.
We will only consider reductions of this simple
form.
59
Cook vs Karp Reductions
Palg poly Poracle
Cook Reduction Design any fast algorithm for
Palg using a supposed fast algorithm for
Poracle as a subroutine.
Karp Reduction The algorithm for Palg calls
that for Poracle only once Yes ? Yes
No ? No
60
Reductions
We give a fast algorithm for Palg using a
supposed fast algorithm for Poracle as a
subroutine.
Is there a fast algorithm for Palg?
Is there a fast algorithm for Poracle?
?
61
Reductions
We give a fast algorithm for Palg using a
supposed fast algorithm for Poracle as a
subroutine.
If there is not a fast algorithm for Palg
If there is a fast algorithm for Palg
?
??
then there is not fast algorithm for Poracle
?
If there is a fast algorithm for Poracle
If there is not a fast algorithm for Poracle
then there is a fast algorithm for Palg
??
?
?
62
Reductions
We give a fast algorithm for Palg using a
supposed fast algorithm for Poracle as a
subroutine.
Conclusions
Palg is at least as easy as Poracle (Modulo
polynomial terms.)
Poracle is at least as hard as Palg (Modulo
polynomial terms.)
The problems have a similar underling structure.
Notation Palg poly Poracle
63
An Application Matching
3 matches Can we do better?
4 matches
Who loves whom.
Who should be matched with whomso as many as
possible matchedand nobody matched twice?
64
Network Flow

• Instance
• A Network is a directed graph G
• Edges represent pipes that carry flow
• Each edge ltu,vgt has a maximum capacity cltu,vgt
• A source node s out of which flow leaves
• A sink node t into which flow arrives

Goal Max Flow
65
Given an oracle for Network Flow, how can you
quickly solve Maching?
66
Reduction
Matching poly Network Flows
67
Reductions
We give a fast algorithm for Palg using a
supposed fast algorithm for Poracle as a
subroutine.
Conclusions
Palg is at least as easy as Poracle (Modulo
polynomial terms.)
Poracle is at least as hard as Palg (Modulo
polynomial terms.)
The problems have a similar underling structure.
Notation Palg poly Poracle
68
NP-Complete Problems
SAT
Lets compare the problems in NP
None are harder than SAT!
NP-Hard
Circuit-Sat Problem Does a circuit have a
satisfying assignment.
GCD
69
NP-Complete Problems

• Problem Pnew is NP-Complete
• Pnew not too hard.
• Pnew ? NP
• Pnew hard.

Pnew
GCD
70
Algorithms Inter-Changeable

Proof by Reductions
71
Given an oracle for Independent Set, how can you
quickly solve Clique?
72
Complement Of G
G
Given a graph G,
G has edge if and only G does not
G
73
Complement Of G
G
Given a graph G,
G has edge if and only G does not
G
74
CLIQUE / INDEPENDENT SET
G
if and only if
G
75
Reduction
Clique poly Indep Set
ltG,kgt
76
Reduction
Clique poly Indep Set
ltG,kgt
Proof of correctness Our oracle says yes to
ltG,kgt iff Old oracle says yes to ltG,kgt iff G
has a k indep. set iff G has a k clique
77
The other direction?
78
Given an oracle for Clique, how can you quickly
solve Independent Set?
Reduction
Indep Set polyClique
79
Reduction
Indep Set poly Clique
ltG,kgt
80
Thus, we can quickly reduce clique problem to an
independent set problem and vice versa. There
is a fast method for one if and only if there is
a fast method for the other.
81
12 Steps

• To prove that Pnew is NP-Complete we do a
  reduction.
• This is generally taught using 3 or 4 steps.
• After collecting lots of wrong answers, I have
  broken reductions into 12 steps.
• 1) I am addicted to solving NP-Complete
  Problems
• 2) I trust in my higher power to help

82
12 Steps

• To prove that Pnew is NP-Complete we do a
  reduction.
• This is generally taught using 3 or 4 steps.
• After collecting lots of wrong answers, I have
  broken reductions into 12 steps.
• Understand them.
• Learn them.
• Follow them carefully.
• Keep the steps separate, because errors happen
  when they are intertwined.

83
Step 0 Pnew ? NP
Formal definition Pnew ? NP iff ? poly time
Valid such that Pnew(I) ? S Valid(I,S)

• Design poly-time algorithm Valid(I,S)
• which determines whether a given solution S
• is valid for a given instance I.

84
Step 1 What to reduce it to
Parbitrary NP poly
Pis NP-comp poly Pnew
Choose a problem Pis NP-comp that is as similar
to Pnew as possible and that is already known to
be NP-Complete.
85
Step 2 What is what
Pnew
Pis NP-comp
Independent Set
Clique
Inew
Snew
Iis NP-comp
Sis NP-comp
86
Step 3 Direction of reduction Code

• Reduce Pnew to Pis NP-comp or Pis NP-comp to
  Pnew?
• People get this wrong.
• Dont memorize it. Go through it each time.
• I want to prove Pnew is hard
• hence that Pis NP-comp ?poly Pnew
• hence that Pis NP-comp is easy
• hence I need to design an algorithm for Pis
  NP-comp
• hence I start with an input instance Iis NP-comp
  for Pis NP-comp
• to get help, I must map it to an instance Inew
  for Pnew

Pis NP-comp poly Pnew
87
Step 3 Direction of reduction Code
88
Step 4 Look for similarities
Clique
Independent Set
Both instances are graphs nodes edges An
clique solution is a subset of the nodes with
edges. An Ind Set solution is a subset of the
nodes with out edges.
89
Step 4 Look for similarities
Schedule each course.Courses that conflict cant
be at same time.
Colour each node.Nodes with lines between them
must have different colours.

Instance
Solution
90
Mapping Steps 5-9
Palg
Poracle
poly
Ialg
Ioracle
Salg
Soracle

• Palg poly Poracle
• Step 5 Ioracle InstanceMap(Ialg)
• Steps 6,7 Salg SolutionMap(Soracle)
• Steps 8,9 Soracle ReverseSolutionMap(Salg)

91
Mapping Steps 5-9
Not done here
Palg
Palg
?poly
Poracle
Poracle
poly
Ialg
Ialg
Ioracle
Ioracle
Salg
Salg
Soracle
Soracle

• Palg poly Poracle
• Step 5 Ioracle InstanceMap(Ialg)
• Steps 6,7 Salg SolutionMap(Soracle)
• Steps 8,9 Soracle ReverseSolutionMap(Salg)

• Poracle poly Palg
• Step 5 Ialg InstanceMap(Ioracle)
• Steps 6,7 Soracle SolutionMap(Salg)
• Steps 8,9 Salg ReverseSolutionMap(Soracle)

92
Step 5 Instance Map
93
Step 5 Instance Map
Ialg
Yes mapped to Yes No to No
Ioracle
ok mapped to twice
ok not mapped to
94
Step 5 Instance Map
Ialg
k4
Yes instances to Yes instances
Ioracle InstanceMap(Ialg)
95
Step 5 Instance Map
Ialg
k5
No instances to No instances
Ioracle InstanceMap(Ialg)
k5
96
Step 5 Instance Map
Ialg
InstanceMapdoes not know if Ialg is a
Yesinstance.
Ioracle
97
Step 5 Instance Map
Ialg
InstanceMapdoes not know if Ialg is a
Yesinstance.
Ioracle
A useful mappingbut not poly-time.
98
Step 6 Solution Map
99
Step 6 Solution Map
Ialg
Ioracle InstanceMap(Ialg)
Every solution Soracle needs to be
considered. Not just those you intended when you
created I oracle.
100
Step 6 Solution Map
Ialg
Ioracle InstanceMap(Ialg)
To be safe,cast a bigger net.
101
Step 7 Valid to Valid
Ialg
Ioracle InstanceMap(Ialg)
If Soracle is valid solution for Ioracle,then
Salg is valid solution for Ialg
Valid
102
Step 6 Solution Map vsStep 7 Valid to Valid

• It is best to separate the steps of
• defining the entire solution Salg being mapped
  toin a well defined and clear way.
• the proof that Salg is a valid solution.
• A false proof
• defines part of Salg
• proves that this makes some aspect of Salg valid
• later to makes some other aspect of Salg valid,
• it redefines the same part of Salg in a
  inconsistent way.

103
Step 6 Solution Map Step 7 Valid to Valid

• Algoracle says yes to Ioracle
• Ioracle has a valid solution Soracle
• Ialg has a valid solution Salg
• Ialg is a yes instance.

104
Step 8 ?? Step 9 ?

• Algoracle says no to Ioracle
• Ioracle does not have a valid solution Soracle
• Ialg does not have a valid solution Salg
• Ialg is a no instance.

No witness

• Ioracle has a valid solution Soracle
• ? Ialg has a valid solution Salg

Soracle ReverseSolutionMap(Salg)
Not part of code, but of proof.
105
Step 8 Reverse Solution Map
Ialg
Ioracle InstanceMap(Ialg)
106
Step 68 Forward vs Reverse Solution Map
Salg SolutionMap(Soracle)
Soracle ReverseSolutionMap(Salg)
Salg
In this case, bijective
Soracle
107
Step 68 Forward vs Reverse Solution Map
Salg SolutionMap(Soracle)
Salg
ok not mapped to
ok mapped to twice
Even the unexpected solutionsneed to be mapped.
Soracle
108
Step 8 Reverse Solution Map
Ialg
Ioracle InstanceMap(Ialg)
109
Step 9 Valid to Valid
Ialg
Valid
Ioracle InstanceMap(Ialg)
If Salg is valid solution for Ialg,then
Soracle is valid solution for Ioracle
110
Step 10 Working Algorithm
Our Algalg says yes to Ialg iff old
Algoracle says yes to Ioracle iff Ioracle has a
valid solution Soracle iff Ialg has a valid
solution Salg iff Ialg is a yes instance.
111
Step 11 Running Time
Algalg(Ialg) is poly time if the following are
poly time
Algoracle(Ioracle)
Assumed
Ioracle InstanceMap(Ialg)
Salg SolutionMap(Soracle)
112
Palg poly Poracle
or Palg Poracle
This will prove that Cir-SAT is NP-Complete.
113
Reduction
Any NP-problem poly Cir-SAT
I
114
We need to solve some unknown NP-Problem. What do
we know about it?
Formal definition Parbitrary ? NP iff ? poly
time Valid such that Parbitrary(I) ? S
Valid(I,S)
We have a poly-time alg Valid(I,S) to test
whether or not S is a valid solution for I.

Valid
115
Reduction
Any NP-problem poly Cir-SAT
I
116
The Circuit Satisfiability Problem
An instance is a circuit C.
An solution is an assignment X F,T,F.
C(X) evaluates to T or F.
One bit outputNo feedback
117
The Circuit Satisfiability Problem
An instance is a circuit C.
An solution is an assignment X F,T,F.
An valid solution has C(X) True.
Given a circuit, does it have a satisfying
assignment?
118
The Circuit Satisfiability Problem
Turing (and friends) prove that any algorithm
computed by a JAVA program in poly-time can be
computed by a Turing Machine in poly-time.
Very Powerful
Cook proves that any algorithm computed by a
Turing Machine in time T(n) can be computed by
a family of circuits of size T(n)2.
But clearly, circuits compute.
119
Reduction
Any NP-problem poly Cir-SAT
I
I build a circuit VI(S) equivalent to Valid(I,S)
120
Reduction
Any NP-problem poly Cir-SAT
I
I build a circuit VI(S) equivalent to Valid(I,S)
121
12 Step Program

• Let be more formal.
• And flow the 12 steps.

122
Step 0 Cir-SAT ? NP

• Design poly-time algorithm ValidCir-SAT(C,X)
• which determines whether a given assignment X
• is valid for a given circuit C.
• Easy Evaluate C(X).

123
Step 1 What to reduce it to
Palg poly Cir-Sat
Choose a problem Pis NP-comp that is as similar
to Cir-Sat as possible and that is already known
to be NP-Complete.
Any NP-problem poly Cir-SAT
We have a poly-time alg Valid(I,S) to test
whether or not S is a valid solution for I.

124
Step 2 What is what
Pnew
Parbitrary
Circuit-Sat
some NP problem
Inew
Snew
Iarbitrary
Sarbitrary
S
I
125
Step 3 Direction of reduction Code
Given oracle for Cir-Sat, we need to be able to
solve any NP-Complete problem.
126
Step 4 Look for similarities
S
I
?
127
Step 5 Instance Map
128
Step 5 Instance Map
I ? circuit
We have a poly-time alg Valid(I,S) to test
whether or not S is a valid solution for I.

• Let Validn(I,S) be a circuit
• I is a bit string representing an instance I.
• S is a bit string representing a solution S.
• Outputs T if S is an encoding of a valid
  solution of I.

Eg Clique
I
S ? k
129
Step 5 Instance Map
I ? circuit
Given an instance I
S
I
hard wired
Outputs T if S is an encoding of a valid
solution S of I
Validn(I,S)
130
Step 6 Solution Map
131
Step 6 Solution Map
S ? assignment
XT,F,F,T,F,T
If X is not a bit string representing a
solutionthen what ever
132
Step 6 Solution Map
S ? assignment
XT,F,F,T,F,T
133
Step 6 Solution Map Step 7 Valid to Valid

• Algoracle says yes to Ioracle
• Ioracle has a valid solution Soracle
• Ialg has a valid solution Salg
• Ialg is a yes instance.

134
Step 7 Valid ? Valid
VI(X) T
135
Step 8 ?? Step 9 ?

• Algoracle says no to Ioracle
• Ioracle does not have a valid solution Soracle
• Ialg does not have a valid solution Salg
• Ialg is a no instance.

No witness

• Ioracle has a valid solution Soracle
• ? Ialg has a valid solution Salg

Soracle ReverseSolutionMap(Salg)
Not part of code, but of proof.
136
Step 8 Rev. Sol. Map
solution ? assignment
solution S
137
Step 9 Valid ? Valid
S is a valid solution of I.
138
Reduction
Any NP-problem poly Cir-SAT
i.e. ? S, S is a valid solution for I
i.e. ? X, VI(X)
139
Palg poly Poracle
or Palg Poracle
Circuit Satisfiability
Graph Colouring
Scheduling
140
3-Col
Graph Col Given graph G k
can G be coloured with k colours? 3-Col
Given graph G can G be
coloured with 3 colours?
3-Col poly Graph Col
If you can decide whether you can colour with
k colours, then you can decide whether you can
colour with 3 colours.
141
Palg poly Poracle
or Palg Poracle
Circuit Satisfiability
Graph Colouring
Scheduling
142
3-SAT

• Cir Sat Given circuit C
• does it have a satisfying assignment X?
• 3-SAT Given an expression
• A circuit consisting of a big AND of clauses
• Each clause is the OR of at most 3 literals
• Each literal is a variable or its negation.
  
• does it have a satisfying assignment X?

143
3-SAT
Cir Sat Given circuit C does it have a
satisfying assignment X? 3-SAT Given an
expression does it have a satisfying
assignment X?
3-Sat poly Cir Sat
If you can decide whether if any circuit is
satifiable, then you can decide whether a
circuit that happens to be an expression is
satisfiable.
144
Palg poly Poracle
or Palg Poracle
3-SAT
3-Col
Circuit Satisfiability
Graph Colouring
Scheduling
3-Col poly 3-SAT
145
Step 5 Instance Map
Graph G ? Expression V
Given an instance G
Encoding C of C u is a node r is a colour
xltu,rgt T if node u is colour r.
146
Step 5 Instance Map
Graph G ? Expression V
Given an instance G
Encoding C of C u is a node r is a colour
xltu,rgt T if node u is colour r.
clauses with 2 literals
147
Step 5 Instance Map
Graph G ? Expression V
Given an instance G
Encoding C of C u is a node r is a colour
xltu,rgt T if node u is colour r.
clauses with 3 literals
148
Reduction
3-Col poly 3-SAT
i.e. ? C, C is a valid 3-colouring for G
i.e. ? X, VG(X)
149
Palg poly Poracle
or Palg Poracle
3-SAT
3-Col
Circuit Satisfiability
Graph Colouring
Scheduling
This will prove that 3-Col is NP-Complete.
And so are Scheduling,
Graph-Colouring, and 3-SAT.
150
Two Different Problems
151
Two Different Problems
Given a circuit,find a satisfying assignment.
Ciruit Satisfiablity poly 3-Col
This will prove that 3-Col is NP-Complete.
152
The Circuit Satisfiability Problem
Very Powerful
JAVA program Turing Machine Circuits
Clearly, circuits compute.
153
The Circuit Satisfiability Problem
An instance is a circuit C.
An solution is an assignment X F,T,F.
C(X) evaluates to T or F.
One bit outputNo feedback
154
The Circuit Satisfiability Problem
An instance is a circuit C.
An solution is an assignment X F,T,F.
An valid solution has C(X) True.
Given a circuit, does it have a satisfying
assignment?
155
Step 0 3-Col ? NP

• Design poly-time algorithm Valid3-Col(G,C)
• which determines whether a given 3-colouring C
• is valid for a given graph G.
• Easy Check that each edge is bi-chromatic.

156
Step 1 What to reduce it to
Palg poly 3-Col
Choose a problem Pis NP-comp that is as similar
to 3-Col as possible and that is already known
to be NP-Complete.
Pis NP-comp Cir-SAT
157
Step 2 What is What

• 3-Colouring
• instance graph
• solution green/red/blue to each node.

• Circuit Problem
• instance circuit of gates
• solution true/false to each variable

158
Step 3 Direction of reduction Code
159
Reduction
Circuit poly 3-Colouring
160
Step 4 Look for Similarities

• 3-Colouring
• instance graph
• solution green/red/blue to each node.

• Circuit Problem
• instance circuit of gates
• solution true/false to each variable
• true/false to internal wires

of gadgets
?
161
A Graph Named Gadget
162
Graph Colouring
Colour each node. Nodes with lines between them
must have different colours. Could you do it with
2 colours?
163
Graph Colouring
Colour each node. Nodes with lines between them
must have different colours.
Could you do it with 3 colours?
Yes.
How do we know?
164
Y
X
Output
variableswires nodes
gate gaget
165
F
T
Y
X
Output
true green false red ? yellow
T
166
F
T
Y
X
Output
true green false red ? yellow
167
F
T
Y
X
?
Output
gate function gadget function
168
F
T
Consider your colouring
Y
X
?
Output
gate function gadget function
169
F
T
Consider your colouring
Y
X
T
F
Output
gate function gadget function
T
170
F
T
Consider your colouring
Y
X
F
T
Output
T
F
gate function gadget function
T
171
F
T
Consider your colouring
Y
X
T
T
Output
T
T
gate function gadget function
T
172
F
T
Consider your colouring
Y
X
Or - Gadget
Output
gate function gadget function
T
173
F
T
Proof
Y
X
T
F
Or - Gadget
Output
gate function gadget function
T
174
F
T
Proof
Y
X
F
F
Or - Gadget
Output
F
F
gate function gadget function
F
175
NOT - Gadget
176
Reduction
Circuit poly 3-Colouring
177
Step 5 Instance Map
178
Step 5 Instance Map
circuit ? graph
179
Step 5 Instance Map
circuit ? graph
x
y
z
NOT
OR
OR
Output
variableswires nodes
180
Step 5 Instance Map
circuit ? graph
x
y
z
NOT
OR
OR
Output
One pallet for defining truth.
gate gaget
181
Step 5 Instance Map
circuit ? graph
y
z
x
x
y
z
NOT
The Graph Named OR-Gadget
OR
Output
gate gaget
182
The Graph Named OR-Gadget
183
Step 5 Instance Map
circuit ? graph
y
z
x
x
y
z
NOT
OR
Output
gate gaget
184
Step 5 Instance Map
circuit ? graph
y
z
x
x
y
z
OR
The Graph Named NOT-Gadget
OR
Output
gate gaget
185
(No Transcript)
186
Step 5 Instance Map
circuit ? graph
y
z
x
x
y
z
OR
OR
Output
gate gaget
187
Step 5 Instance Map
circuit ? graph
y
z
x
x
y
z
NOT
OR
Output
The Graph Named OR-Gadget
gate gaget
188
The Graph Named OR-Gadget
189
Step 5 Instance Map
circuit ? graph
y
z
x
x
y
z
NOT
OR
Output
gate gaget
190
Step 5 Instance Map
circuit ? graph
One extra edge from Falseto Output
191
Step 5 Instance Map
circuit ? graph
Done
192
Step 5 Instance Map
Ialg circuit
Yes mapped to Yes No to No
Ioracle graph
ok not mapped to
193
Step 6 Solution Map
194
Step 6 Solution Map
assignment ? colouring
y
z
x
Assume these are the colours of these nodes.
Output
195
Step 6 Solution Map Step 7 Valid to Valid

• Algoracle says yes to Ioracle
• Ioracle has a valid solution Soracle
• Ialg has a valid solution Salg
• Ialg is a yes instance.

196
Step 7 Valid ? Valid
y
z
x
x
y
z
OR Gadget
F
Output
Output
197
Step 7 Valid ? Valid
y
z
x
x
y
z
Not Gadget
F
Output
Output
198
Step 7 Valid ? Valid
y
z
x
x
y
z
F
F
Output
OR Gadget
Output
199
Step 7 Valid ? Valid
y
z
x
x
y
z
F
F
Output
F
One extra edge from Falseto Output
Oops this is a bad coloring
Output
200
Step 6 Solution Map
assignment ? colouring
y
z
x
Assume these are the colours of these nodes.
Output
201
Step 7 Valid ? Valid
y
z
x
x
y
z
T
Output
Output
202
Step 7 Valid ? Valid
y
z
x
x
y
z
F
T
Output
T
One extra edge from Falseto Output
Output
T
Cant be F
203
Step 7 Valid ? Valid
y
z
x
x
y
z
F
T
Output
T
T
valid colouring ? satisfying
assignment
Output
T
204
Step 6 Solution Map Step 7 Valid to Valid

• Algoracle says yes to Ioracle
• Ioracle has a valid solution Soracle
• Ialg has a valid solution Salg
• Ialg is a yes instance.

205
Step 8 ?? Step 9 ?

• Algoracle says no to Ioracle
• Ioracle does not have a valid solution Soracle
• Ialg does not have a valid solution Salg
• Ialg is a no instance.

No witness

• Ioracle has a valid solution Soracle
• ? Ialg has a valid solution Salg

Soracle ReverseSolutionMap(Salg)
Not part of code, but of proof.
206
Step 8 Rev. Sol. Map
assignment ? colouring
y
z
x
x
y
z
Output
Output
207
Step 9 Valid ? Valid
y
z
x
x
y
z
Output
Edges in Gadgetsbi-chromatic
Output
208
Step 9 Valid ? Valid
y
z
x
x
y
z
Output
Extra Edge?
Extra Edge bi-chromatic
Output
Must be T
209
Step 9 Valid ? Valid
y
z
x
x
y
z
Output
satisfying assignment ? valid
colouring
Output
210
Step 5-9
y
z
x
x
y
z
F
T
Output
T
T
Satisfiability of this circuit
3-colourability of this graph
Output
211
Satisfiability of this circuit
3-colourability of this graph
212
Reduction
Circuit poly 3-Colouring
213
Palg poly Poracle
or Palg Poracle
3-SAT
3-Col
Circuit Satisfiability
Graph Colouring
Scheduling
Clique
Independent Set
This will prove that 3-Col is NP-Complete.
And so are Scheduling,
Graph-Colouring, and 3-SAT.
214
Palg poly Poracle
or Palg Poracle
3-SAT
3-Col
Circuit Satisfiability
Graph Colouring
Scheduling
Clique
Independent Set
True but we do will not do this here.
215
Palg poly Poracle
or Palg Poracle
3-SAT
3-Col
Circuit Satisfiability
Graph Colouring
Scheduling
Clique
Independent Set
If we can solve one of these quickly, then we
can solve all NP-problems quickly.
216
Palg poly Poracle
or Palg Poracle
If we can prove one takes exponential time,
then all NP-complete problems take exponential
time.
217
Yes If we can solve Sat fast, we can solve
Network Flow fast, because we can solve Network
Flow fast.
218
?
No We can solve Network Flow fast, but we cannot
solve Sat fast.
Network Flows
219
Yes We use the fast alg for Network Flow to
give a fast alg for Bipartite Matching.
Network Flows
220
?
Math Truth
Network Flows
Halting problem ?poly Math Truth ? Math Truth ?
Undecidable.
Bipartite Matching
221
?
Yes If we can solve Math Truth fast, we can
solve Sat fast.
Math Truth
Network Flows
Bipartite Matching
222
Boss assigns task

• Given courses students want, schedule them
  to minimize conflicts.

223
Your answer

• Sorry. Very likely no fast algorithm exits that
  always finds best schedule.

Else have fast algorithm for all NP-problems.
Worthy of wealth Nobel prize
224
Cryptography
Sometimes it is useful to know that a problem is
computationally hard to do.
For example, knowing no one will break into your
secret documents.
225
Cryptography
I chose two primes p and q and multiply them N
pq. I publish N to the world.
I have a message M to send to him. I use N to
encode it. Code Encode(M,N)
Knowing p and q, I can decode the message. M
Decode(Code,p,q)
226
Identifying Oneself
I am the guy who knows p and q.
Prove it. I will encode a message for you. Code
Encode(M,N)
Knowing p and q, I can decode the message. M
Decode(Code,p,q)
227
Countable and Uncountable Infinity

• A finite set contains some integer number of
  elements.
• , 4,
• The size of an infinite set is bigger than any
  integer.
• The set of integers Z 1,2,3,4,
• The set of fractions Q 1/2, 2/3, ...
• The set of reals R 2.34323, 34.2233,
  ?, e,
• Are these infinite sets the same size?
• Two sets have the same size if there is a
  mapping between them.

1, 2, 3
3
228
Countable and Uncountable Infinity

ltlt
Integers
Fractions
Reals
Each defined by an infinite string
Each defined by a finite string
Each defined by a finite string
ltlt

• Set of Algorithms
• Each defined by a finite string
• Countable in size

• Set of Computational Problems
• Set of mappings from 0,1 strings
  to true false.
• Each defined by an infinite string
• uncountable in set

229
Many Uncomputable Problems
Some problem
Games
3-Col
GCD
230
Halting Problem

• You boss asks
• Given a computer program, will it halt?

231
Halting Problem

• loop a 1 1,000,000
• s sa
• end loop

Halts
loop a gt 0 s sa end loop
Does not halt
232
Halting Problem

• loop a,b,c,z gt 2
• exit when az bz cz
• end loop

Proof of not halting Proof of Fermats Last
Theorem
Chaotic behavior not knowing what it will do
without doing it.
loop chaotic behavior exit when special
event end loop
How can we know it will not halt without running
it forever?
233
Halting Problem

• Sorry. No algorithm exists that tells you for
  every computer program whether it halts!

234
Infinite Sets can be counter-intuitive

• For every integer there is a bigger integer,
• for every finite set, there is a bigger integer,
• but there is not a biggest integer

Ackermans-1(1,000,000,000)
Numbers used by Mathematicians
Ackermans-1(5)
1,000,000,000
Numbers used by TAs.
1,000,000
Numbers used by 1st year students.
82
1, 2 , 3,
235
Infinite Sets can be counter-intuitive

• For every finite set of programs,
• there is a more complex program,
• but there is not a most complex program

loop chaotic behavior exit when special
event end loop
loop a,b,c,z gt 2 exit when az bz cz end loop
Programs used by Mathematicians
loop a gt 0 s sa end loop
Programs used by TAs.
loop a 1 1,000,000 s sa end loop
Programs used by 1st year students.
236
Halting Problem

• Programs can take themselves as inputs. i.e.
  Program P running on input P.
• Compliers can compile themselves.
• Editors can edit the code for the editor.

237
Halting Problem
Suppose there was a working algorithm Halt(P,I)
whether program P halts on input I Construct
from it another algorithm Nhalt(P)
if(Halt(P,P)) loop forever else halt halts
? program P does not halt on input P. Paradox
Nhalt(Nhalt) ?
program Nhalt halts on input Nhalt ? program
Nhalt does not halt on input Nhalt
Contradiction!
238
Halting Problem

• Sorry. No algorithm exists that tells you for
  every computer program whether it halts!

Or any other useful thing about what the program
does.
Does program P output 5 on input I
239
History of Proofs
And it was good.
240
History of Proofs
Clearly everything is either true or false. Goal
Design a proof system that proves
everything. Result Prove that God exists
241
History of Proofs

• A proof system consists of
• A finite set of axioms (Statements assumed to
  be true)

Eg. 1) Any two points can be joined by
a straight line . 5) Parallel
lines never meet.
242
History of Proofs

• A proof system consists of
• A finite set of axioms (Statements assumed to
  be true)
• A finite set of rules for proving one statement
  from previously proved theorems.

Eg If statements A and A ? B have been
proved, then statement B follows.
243
History of Proofs
To the ancients, the parallel axiom 5)
Parallel lines never meet.seemed less obvious
than the others. (God would want a cleaner
world) They wanted to prove it from the other
four. By 1763 at least 28 different proofs had
been published. They were all false!!!
Euclid (300 BC)
There exists Non-Euclidian (curved) worlds in
which first four axioms are true and the fifth is
false. Examples Earth Our universe.
244
History of Proofs
Clearly everything is either true or false.
Oops
Some things have no proof and whether true or
false depends on your world view.
245
History of Proofs
Clearly everything is either true or false.
Consider only statements in Number Theoryi.e.
statements about the integers.
eg I ? a,b,c,r ? 3 arbr ?cr
Clearly each such statement is either true or
false.
Goal Design a proof system that
proves/disproves all of these.
246
History of Proofs
Goertels Incompleteness Theorem
Some things have no proof and whether true or
false depends on your world view. For every
proof system S,there are math statements Iwhich
are either not provedor proved
incorrectly! Proof (CS Theory) If there was such
an S then the problem Math Truth would be
computable. But it is not.
247
Math Truth

• Problem Math Truth
• Instance A math statement I eg. I ?
  a,b,c,r ? 3 arbr ?cr
• Solution A proof P either of I or of I

I can check if it is a valid proof within S.
Doesnt this mean the problem is in NP?
No, because the proofs may be really big.
248
Math Truth
Here is an algorithm for Math Truth (Even without
me)
Alg(I) Loop through all proofs P if P is a
proof in S of I or of ?I exit
Eventually this program will find the proof and
stop.
249
Math Truth wrt Proof System
Halting
Math Truth
Games
3-Col
GCD
250
Math Truth
Here is an algorithm for Math Truth (Even without
me)
Alg(I) Loop through all proofs P if P is a
proof in S of I or of ?I exit
Wait!
251
Math Truth
Here is an algorithm for Math Truth (Even without
me)
Alg(I) Loop through all proofs P if P is a
proof in S of I or of ?I exit
But what if P has no proof? Then this algorithm
runs for ever. Will this algorithm ever stop? It
reminds me of the Halting problem!
252
Math Truth
Here is an algorithm for Math Truth (Even without
me)
Alg(I) Loop through all proofs P if P is a
proof in S of I or of ?I exit
But what if P has no proof?
253
Math Truth
Alg(I) Loop through all proofs P if P is a
proof in S of I or of ?I exit
254
Math Truth wrt Proof System
Halting
Math Truth wrt Proof System
Games
?
Math Truth
3-Col
Contradiction!
And hence, there is no such proof system S!!!
GCD
255
Math Truth wrt Proof System
Halting
Math Truth wrt Proof System
Wait. We must still prove
Math Truth Uncomputable
Games
Maybe it is similar to the Halting problem
3-Col
And hence, there is no such proof system S!!!
GCD
256
Halting problem ?poly Math Truth
ltP,Igt

257
Halting problem ?poly Math Truth
Math statement Program P halts on input I
? C, C is an integer encoding a
valid halting computation for
program P on input I
Mark Headwith digit 2
258
Halting problem ?poly Math Truth
Math statement Program P halts on input I
? C, C is an integer encoding a
valid halting computation for
program P on input I
A valid computation of a TM

• Time state Tape Contents Head 2
• 1 102 2,0,1,1,0,0,1,1
• 2 10112 1,2,1,1,0,0,1,1
• i 11012 0,0,1,1,2,0,0,1,1,0
• i1 10102 0,0,1,2,1,1,0,1,1,0
• T 1102 2,0,0,1,1,1,0,1,0,1,0

Mark Headwith digit 2
Separate blocks with digits 3,4


259
Halting problem ?poly Math Truth
Math statement Program P halts on input I
? C, C is an integer encoding a
valid halting computation for
program P on input I
A valid computation of a TM

• Time state Tape Contents Head 2
• 4 10 3 2,0,1,1,0,0,1,1
• 4 1011 3 1,2,1,1,0,0,1,1
• 4 1101 3 0,0,1,1,2,0,0,1,1,0
• 4 1010 3 0,0,1,2,1,1,0,1,1,0
• 4 110 3 2,0,0,1,1,1,0,1,0,1,0

Separate blocks with digits 3,4
Remove ,


260
Halting problem ?poly Math Truth
Math statement Program P halts on input I
? C, C is an integer encoding a
valid halting computation for
program P on input I
A valid computation of a TM

• Time state Tape Contents Head 2
• 4 10 3 2 0 1 1 0 0 1 1
• 4 1011 3 1 2 1 1 0 0 1 1
• 4 1101 3 0 0 1 1 2 0 0 1 1 0
• 4 1010 3 0 0 1 2 1 1 0 1 1 0
• 4 110 3 2 0 0 1 1 1 0 1 0 1 0

Remove ,
Merge Digits


261
Halting problem ?poly Math Truth
Math statement Program P halts on input I
? C, C is an integer encoding a
valid halting computat