Overview of the theory of computation

1
Overview of the theory of computation
Episode 3

• Turing machines
• The traditional concepts of computability,
  decidability and recursive enumerability
• The limitations of the power of Turing machines
• The Church-Turing thesis
• Mapping reducibilty
• Turing reducibility
• Kolmogorov complexity

0
2
Components of a Turing machine (TM)
3.1
2 0 - - - - - - -
Input tape
Read-only
Read-write
4 0 0 - 1 2 - -
Work tape
4 0 0 - - - - - -
Output tape
No direct access
Control (transition function)
- blank symbol
Has a finite number of states, two of which,
Start and Halt, are special.
3
How a TM works
3.2
2 0 - - - - - - -
Input tape
- - - - - - - - -
-
Work tape
- - - - - - - - -
Output tape

• At the beginning, the machine is in its Start
  state, the input tape has an input on it,
• the other tapes are blank, and the two scanning
  heads are in their leftmost positions.
• Such a situation is shown above, with input being
  20.
• After the computation starts, the machine goes
  from one state to another, writes on the
• work tape, and moves its scanning heads left or
  right according to its transition function,
• which precisely prescribes what to do in each
  particular case, depending on the current
• state and the contents of the two scanned cells
  (on the input and work tapes).
• If and when the machine enters the Halt state,
  it halts, and whatever string w is to
• the left of the work-tape scanning head, will be
  automatically copied onto the output
• tape. Such a string w is considered to be the
  output of the machine. Slide 3.1 shows
• such a situation, with output 400.

4
Variants of Turing machines
3.3

• There are numerous reasonable variations of
  the Turing machine
• model, and they all turn out to be equivalent.
  For example
• More often than not, there is just a single tape
  for everything (input,
• output, work).
• Some models allow multiple work tapes.
• The tape can be infinite in both directions
  (have no beginning).
• And so on.

One of the advantages of the particular model
we have chosen here is that it can be easily
adapted to cases where infinitely many inputs and
outputs are allowed. In such cases, entering the
Halt state will only have the effect of sending
a (next) output to the output tape, but otherwise
the machine would not halt.
2 0 - 3 - 1 0 - 2
- ...
Input Output
4 0 0 - 9 - 1 0 0 -
...
5
Computability
3.4
Definition 3.1. Let f A?B be a function.
(a) We say that a TM M computes f iff, for
every string w?A, whenever M receives
w as input, M (halts and) outputs string
v such that v f(w). (b) When such a
machine M exists, we say that f is computable.
There is no need to separately consider
functions with more than one (but finitely many)
arguments, as tuples of strings can be encoded
through (just) strings. For example, the
function f(x,y)xy can be thought of as a
single-argument function that returns string 3
on input string (2,1), returns string 20 on
input string (8,12), etc.


6
Decidability
3.5
Definition 3.2. Let S be a set of strings
over a given alphabet A. (a) We say that a
TM M decides S iff, for every string w?A,
whenever M receives w as input, M (halts and)
outputs 1 (or yes, or
accept) if w?S 0 (or no,
or reject) if w?S. (b) When such a machine
M exists, we say that S is decidable.
As every set S can be thought of as and
identified with its characteristic function,
decidability is just a special case of
computability. The concept of decidability
extends to predicates and relations, as the
latter can be thought of as sets.


7
Recursive enumerability
3.6
Definition 3.3. Let S be a set of strings over a
given alphabet A. (a) We say that a TM M
recognizes S iff, for every string w?A,
whenever M receives w as input if
w?S, then M (halts and) outputs yes
if w?S, then M either (halts and) outputs
something else (such as no), or never
halts. (b) When such a machine M exists, we say
that S is recursively enumerable.
Again, this definition extends from sets to
predicates and relations.


8
Computational problems in the traditional sense
3.7
Computational problems in the traditional sense,
as established in the traditional theory of
computation, are nothing but functions, sets,
predicates or relations (to be computed or
decided or recognized). We will follow that
tradition, and often say problem instead of
function, relation, etc. For example, we
may refer to the function f(x)x2 as the problem
of finding the square of a given number, and
refer to the set (predicate, relation) x x is
an even number as the problem of telling
whether a given number is even, or the
evennees problem, etc.


9
The limitations of the power of TMs
3.8
Alas, not all reasonable problems can be handled
by TMs. Examples of
undecidable or incomputable problems
1. The acceptance problem for Turing machines,
i.e. the problem of telling whether a given
Turing machine accepts (meaning outputting yes)
a given input.
2. The halting problem for Turing machines, i.e.
the problem of telling whether a given Turing
machine ever halts on a given input.
3. The problem of finding the Kolmogorov
complexity of a given number.
4. The problem of telling whether a given
polynomial equation has integral roots.
5. The problem of first-order logical validity.
Examples of problems that are not
recursively enumerable
6. The complements of the above problems.
7. The problem of second-order logical validity.
8. The problem of telling whether a given formula
of arithmetic is true.


10
The Church-Turing thesis
3.9

Turing machines
Algorithms
Of course, here should be understood not in
the strict sense, but in the sense of equivalence
of power. That is, according to the
Church-Turing thesis, the problems that have
algorithmic solutions also have Turing machine
solutions (deciders, recognizers, etc.), and
vice versa. Thus, the existence of undecidable
or incomputable problems means the existence of
problems that have no algorithmic solutions,
problems that no computers can ever handle.


11
Definition of mapping reducibility
3.10
Definition 3.4. Let A and B be sets of
strings over an alphabet ?. (a) We say that a
function f ??? is a mapping reduction from A
to B iff f is computable and, for every w??,
w?A iff
f(w)?B. (b) When such a function f exists,
we say that A is mapping reducible to B, and
write A?mB. In the literature, mapping
reducibility is more often called many-one
reducibility.
?
?
A
B
f
f



12
Definition of mapping reducibility
3.10
Definition 3.4. Let A and B be sets of
strings over an alphabet ?. (a) We say that a
function f ??? is a mapping reduction from A
to B iff f is computable and, for every w??,
w?A iff
f(w)?B. (b) When such a function f exists,
we say that A is mapping reducible to B, and
write A?mB. In the literature, mapping
reducibility is more often called many-one
reducibility.
?
?
A
B
f
f



13
Definition of mapping reducibility
3.10
Definition 3.4. Let A and B be sets of
strings over an alphabet ?. (a) We say that a
function f ??? is a mapping reduction from A
to B iff f is computable and, for every w??,
w?A iff
f(w)?B. (b) When such a function f exists,
we say that A is mapping reducible to B, and
write A?mB. In the literature, mapping
reducibility is more often called many-one
reducibility.
?
?
A
B
f
f



14
Using mapping reducibility for proving
decidability/undecidability
3.11
Theorem 3.5. If A?mB and B is decidable, then A
is decidable.
Proof Let DB be a decider for B and f be a
mapping reduction from A to B. We describe a
decider DA for A as follows.
DA On input w 1. Compute f(w).
2. Run DB on input f(w) and output whatever
DB outputs. ?
(? means end of proof ).
Thus, the decidability of a problem A can be
proven by finding a mapping reduction from A to
some problem B which is already known to be
decidable. Or, the undecidability of a
problem B can be proven by finding a mapping
reduction from A to B, where A is a problem
already known to be undecidable.
Theorem 3.5 remains valid with
recursively enumerable instead of
decidable.



15
A mapping reduction of the acceptance problem to
the halting problem
3.12
For every TM M, let M be the following TM
M On input x
1. Run M on x. 2. If M
outputs yes, accept. 3.
If M outputs anything else, enter an infinite
loop.
Thus,

• If M outputs yes on input x, then M
• If M outputs anything else on input x, then M
• If M never halts on input x, then M
• To summarize, M accepts x iff M

accepts x never halts on x never halts on x halts
on x
Let then f be the function defined by
f(M,w)(M,w).
Is f computable?
Of course.
And, since we have (M,w)?ACCEPTANCE_PROB
LEM iff f(M,w)?HALTING_PROBLEM, f is a
mapping reduction of the acceptance problem to
the halting problem.



16
Definition of Turing reducibility
3.13
An oracle for a set (relation) B is an external
device that is capable of reporting whether any
given string w is a member of B.
An oracle Turing machine (OTM) is a modified
Turing machine that has the additional
capability of querying an oracle.
Example Construct an OTM O with an oracle for
the acceptance problem, such that O decides the
nonacceptance problem (the complement of the
acceptance problem).
O On input (M,w), where M is a TM and w is a
string 1. Query the oracle to determine
whether M accepts w. 2. If the oracle
answers NO, accept if YES, reject.
Definition 3.6. We say that a problem A is Turing
reducible to a problem B, written A?TB, iff
there is an OTM M with an oracle for B, such
that M decides A (or M computes A, if A is a
function rather than a relation).



17
Using Turing reducibility for proving
decidability/undecidability
3.14
Theorem 3.7. If A?TB and B is decidable,
then A is decidable.
Proof. If B is decidable, then we may replace the
oracle for B by an actual procedure that decides
B. Thus we may replace the OTM that (using an
oracle for B) decides A by an ordinary TM that
decides A. ?
Does this proof go through for recursively
enumerable instead of decidable? (see the end
of Slide 3.11)
No! In fact, one can prove the opposite. For
example, as we saw on the previous slide, the
nonacceptance problem is Turing reducible to the
acceptance problem (generally, any problem is
Turing reducible to its complement). And the
acceptance problem is obviously recursively
enumerable (why?). But the nonacceptance problem
is not recursively enumerable (why?). It also
follows from the comment at the end of Slide 3.11
that the nonacceptance problem is not mapping
reducible to the acceptance problem.
Yes!
Does mapping reducibility always imply Turing
reducibility?

Turing reducibility is the weakest form of
reducibility.
18
Turing reductions of the acceptance problem to
the halting problem
3.15
Example Show that the acceptance problem is
Turing reducible to the halting problem.
Solution 1. When receiving the question Does M
accept w?, replace M by M (as described on
Slide 3.12), and ask the oracle whether M halts
on w. Repeat whatever answer you get from the
oracle. ?
Solution 2. When receiving the question Does M
accept w?, ask the oracle if M halts on input
w. If the oracle says No, you also say
No. If the oracle says Yes, start
running (simulating) machine M on input w until
you see it has halted. If its output is Yes,
you also say Yes. Otherwise say No. ?
Why could not we just do simulation as described
in the last paragraph of Solution 2, without any
preliminary use of the oracle?

Because there would be no
guarantee that the simulation would ever end.
19
Kolmogorov complexity
3.16
All Turing machines can be listed in the
lexicographic order of their descriptions
M0, M1, M2, M3, ..., Mi,
.... Number i can thus be considered the code of
the machine Mi.
Definition 3.8. Let m be a natural number. The
Kolmogorov complexity of m is the smallest
number i such that machine Mi outputs m on input
0.
Note More often, Kolmogorov complexity is
defined not as the above number i itself, but as
the size i of that number, i.e. the
logarithm of i. For our purposes, however, this
makes no difference, and we will stick to
Definition 3.8 as it is given.
Importance Kolmogorov complexity can be seen
as a mathematical counterpart of the intuitive
concepts of randomness or amount of
information. The greater the Kolmogorov
complexity of a given object, the more random it
is and the more information it contains.
20
Turing reduction of the Kolmogorov complexity
problem to the halting problem
3.17
Example Show that the Kolmogorov complexity
problem (i.e. the problem of finding the
Kolmogorov complexity of a given number) is
Turing reducible to the halting problem.
Solution. After receiving the question What is
the Kolmogorov complexity of m?, initialize
variable i to 0, and do the following 1.
Ask the oracle if the machine Mi halts on input
0. 2. If the oracle says No, increment i
to i1, and go back to Step 1. 3. If the
oracle says Yes, simulate Mi on input 0 until
it halts. If you see that the output of Mi is m,
return i as your output. Otherwise, increment i
to i1 and go back to Step 1. ?

Unlike the example
on Slide 3.15, here the oracle is queried more
than once. The number of queries, however, can
be shown to be bounded by a certain linear
function of m.
How many times does the above algorithm use the
oracle?
Theorem 3.9. The Kolmogorov complexity problem is
not computable.