Programming

1
The Halting Problem

• Can we design a program that, given any other
  program and its input, tells whether that program
  will halt when run on that input?
• Output of this program, called HALT, when applied
  to program P and input I, is halts if P(I)
  eventually halts
• Output of this program is never halts if P(I)
  never halts
• Theorem (Alan Turing 1937) No computer program
  cansolve the halting problem.

2
Proof by Contradiction

• Assume the hypothesis to be proven is false, i.e.
  there is a computer program that can solve the
  halting problem
• Show that this assumption leads to a
  contradiction
• Do it by creating a program and an input to it
  that generate this contradiction

3
Proof by Contradiction

• Suppose there was a computer program,called
  HALT, which solved the halting problem.
• We can write a new program that uses HALT and
  leads to a contradiction

Program NonConformist (Program P) If ( HALT(P)
never halts ) Then Halt Else Do While (1
gt 0) Print Hello! End While End If End
Program
4
Proof by Contradiction
Program NonConformist (Program P) If ( HALT(P)
never halts ) Then Halt Else Do While (1
gt 0) Print Hello! End While End If End
Program

• Does NonConformist(NonConformist) halt?
• Note It calls HALT(NonConformist)
• Yes? Means HALT(NonConformist) never halts
• No? That means HALT(NonConformist) halts
• Contradiction There exists a program
  (NonConformist) for which the HALT program gives
  the wrong answer

5
Undecidability

• Halting ProblemGiven a computer program P and
  input input
• Output halts if P(input) eventually halts
• Output never halts if P(input) never halts
• Weve shown that the Halting Problem is
  Undecidable no computer program can ever solve
  it, no matter how powerful the computer is.

6
Undecidability

• Many other problems are undecidable, too.
• Moreover, the same ideas that we used to prove
  undecidability can be used to prove a very
  disturbing statement.

7
GÖDELS INCOMPLETENESS THEOREM

• Until Gödel came along, Mathematicians were
  searching for the one true logical framework,
  i.e. a framework in which every true statement
  can be proved.
• Their faith was shattered by Gödel.
• For more on this and its philosophical
  implications, see Gödel, Escher, Bach by Douglas
  Hofstadter.

8
GÖDELS INCOMPLETENESS THEOREM

• In 1931, Gödel stunned the world by proving that
  in any logical framework in which you can express
  basic facts about numbers, there exists a true
  statement that cannot be proved in that
  framework.
• Not every true statement has a proof

9
GÖDELS INCOMPLETENESS THEOREM

• In 1931, Gödel stunned the world by proving that
  in any logical framework in which you can express
  basic facts about numbers, there exists a true
  statement that cannot be proved in that
  framework.
• But wait, if it cant be proved, how do we know
  it is true?

10
GÖDELS INCOMPLETENESS THEOREM

• We can prove that it is true by jumping outside
  the original logical framework to a larger
  logical framework

11
GÖDELS INCOMPLETENESS THEOREM

• We can prove that it is true by jumping outside
  the original logical framework to a larger
  logical framework
• But that larger logical framework also has a true
  statement that cannot be proved!

12
GÖDELS INCOMPLETENESS THEOREM

• We can prove that it is true by jumping outside
  the original logical framework to a larger
  logical framework
• But that larger logical framework also has a true
  statement that cannot be proved!
• and so on, and so on
• No matter when we stop, there will always be some
  true statement that doesnt have a proof!

13
GÖDELS INCOMPLETENESS THEOREM

• Until Gödel came along, Mathematicians were
  searching for the one true logical framework,
  i.e. a framework in which every true statement
  can be proved.
• Their faith was shattered by Gödel.
• For more on this and its philosophical
  implications, see Gödel, Escher, Bach by Douglas
  Hofstadter.

14
More Undecidable Problems?

• Post's Correspondence Problem (PCP)

• An instance of PCP of size s is a finite set of
  pairs of strings (gi , hi) for i 1...s sgt1
  over some alphabet ?.
• A solution is a sequence i1 i2 ... in of
  selections from each set of strings (gi , hi)
  such that the strings gi1gi2 ... gin and hi1hi2
  ... hin formed by concatenation are identical.

15
Sample PCP

• g1 aba h1 abaa
• g2 bbab h2 abab
• g3 baaa h3 a
• g4 a h4 bb
• So, 1,3,1,2 would correspond to
• aba baaa aba bbab from gs
• abaa a abaa abab from hs
• Not a solution!

16
Another Sample PCP

• g1 aba h1 abaa
• g2 bbab h2 abab
• g3 baaa h3 a
• g4 a h4 bb
• 1,4,2,1,3 corresponds to
• aba a bbab aba baaa from g
• abaa bb abab abaa a from h
• Solution!

17
PCP is undecidable?

• PCP shown to be undecidable by Post in 1946.
• What about PCP with limited-size inputs
• PCP with size 2 has been proved decidable.
• PCP with size 7 has been proved undecidable
• The decidablility of problems with size between 3
  and 6 is still pending.

18
What Computers Cant DoIn Your Lifetime

• Weve now seen examples of problems that
  computers cant solve, even if computers have
  unlimited speed and unlimited time.
• Are there more real world problems (eg that arise
  in business, science, ) that can be solved but
  take far too much time to be solved in practice?

19
Search Problems

• Well focus on what well call search
  problems.(In Computer Science terms, well be
  talking about what are termed NP problems)
• Intuitively, a search problem is a problem where
  you are looking for something which you can
  recognize quickly if you find it.
• Recognizing a good solution is easy
• Problem is finding it

20
Example Coloring
21
Example Coloring with 3 colors

• Suppose we are given a collection of circles
  (nodes).
• Some circles connect to others by edges, forming
  a graph
• Rule No two connected circles can have the
  same color.
• You only have three colors (Green, Red, Yellow)
• Is there a valid coloring?
• Note easy to check validity. Hard to find?

22
Search ProblemsMore Precise Definition

• Intuitively, a search problem is a problem where
  you are looking for something which you can
  recognize quickly if you find it.
• A bit more precisely, we require that there is a
  small circuit that can quickly check the validity
  of a solution.
• For coloring, it would just be a circuit that
  checks that for every pair of connected circles,
  the colors are different.

23
Another Example Traveling Salesperson Problem

• A saleswoman wants to visit n different cities.
• She knows the costs associated with flights
  between each city.
• Can she visit all the cities spending less than
  B in total?
• Note Easy to check that a given flight plan
  visits all the cities and costs less than B.
• Seems hard to find the flight plan

24
Search Problems

• Unfortunately, we dont know how to solve these
  and many other search problems with a computer in
  our lifetimes for large inputs.
• Large graphs, large number of cities
• For many years, computer scientists wondered
  which search problems could be solved, and which
  couldnt.
• But just because computer scientists couldnt
  solve the Coloring problem for 40 years doesnt
  mean it is impossible, right?
• Proof of Fermats Last Theorem took over 300 years

25
Search Problems

• Can we prove that it is impossible to solve them
  quickly for large inputs?
• Unfortunately, not yet
• But now we are much more confident that they
  really are impossible to solve quickly
• How do we make progress toward determining an
  answer?

26
Breakthrough Reduction

• A surprise
• In the early 70s, Cook, Levin, and Karp showed
  us that if we can solve the Coloring problem
  quickly, then we can solve ALL search problems
  quickly!
• But general search problems are defined in terms
  of circuits (that can validate their solutions),
  not colors
• Need to map circuits to graph coloring problems
• Then we can map the circuit for any search
  problem to the corr. graph coloring problem

27
Coloring and Circuits

• Suppose we think of
• Green as meaning True.
• Red as meaning False
• Yellow as meaning nothing

28
What is the circuit for this graph?
Valid coloring for this graph leads to a NOT gate
T
F
So this is a graph for a NOT circuit
29
X Y
F F
F T
T F
T T
30
F
T
Y
X
F
T
X Y
F F
F T
T F
T T
Output
31
F
T
Y
X
X Y
F F
F T
T F
T T
Output
32
F
T
Y
X
X Y
F F
F T
T F
T T
Output
33
F
T
Y
X
X Y
F F
F T
T F
T T
Output
34
F
T
Y
X
X Y
F F
F T
T F T
T T
Output
35
X Y
F F F
F T T
T F T
T T T
36

X Y OR
F F F
F T T
T F T
T T T
37
Coloring !

• And you thought coloring was for kids..
• In fact, we can encode any circuit into a
  collection of connected circles (graph) waiting
  to be colored.
• Any valid coloring for the graph conforms to the
  circuit (which verifies a search problem)
• Thus, we can efficiently reduce any search
  problem to Coloring.
• If we can solve Coloring quickly, we can solve
  any search problem quickly!

38
Many more

• Traveling Salesperson Problem is also as hard
  as any search problem.
• Search Problems with this property are called
  complete problems
• Although we dont know how to prove that they are
  hard, we know that if find a way to solve one of
  them quickly, we can solve all search problems
  quickly!
• Gives us more confidence that they really are
  hard.

39
Recap Hard search problems

• A hard search problem is a problem where
• It is hard to find a solution
• It is easy to check possible solutions for
  validity
• A complete search problem is a problem that is as
  hard as any search problem
• Search problem is believed to be hard because
• No one found a way to solve any of the complete
  search problems quickly

40
What does Quickly mean?

• In time polynomial in the size of the input
• E.g. if one could solve an n-city TSP in time
• n22n5
• or even n20007n19998n25n6
• Then these are a Polynomial-time algorithms
  (quick)
• But if one can only solve it in time
• 2n
• or 2n6n-13
• Then these are Exponential-time algorithms (slow)

41
n2 versus 2n

• n 10
• n2 100
• 2n 1024
• n 100
• n2 10,000
• 2n (1024)10
• gt 1 trillion 1 trillion 1 million
• Hence we call exponential-time algorithms slow

42
Classes of search problems

• In computer-science terminology
• NP All Search Problems
• P Problems we can solve quickly
• We believe that P ? NP, i.e. not every search
  problem can be solved quickly on a computer.

43
NP-Complete Problems

• Coloring is complete
• In particular, we can reduce solving any search
  problem to finding a valid coloring for some
  collection of circles!
• So, if we could solve Coloring quickly, then P
  NP
• Thats why we believe Coloring cant be solved
  quickly by any computer.
• We call such problems NP-Complete.