FORMAL LANGUAGES, AUTOMATA, AND COMPUTABILITY

1
15-453
FORMAL LANGUAGES, AUTOMATA, AND COMPUTABILITY
Read chapter 4 of the book for next time
Lecture9x.ppt
2
REVIEW
3
A Turing Machine is represented by a 7-tuple T
(Q, S, G, ?, q0, qaccept, qreject)
Q is a finite set of states
S is the input alphabet, where ? ? S
G is the tape alphabet, a superset of S ? ? G
? Q ? G ? Q ? G ? L, R is the transition func
q0 ? Q is the start state
qaccept ? Q is the accept state
qreject ? Q is the reject state, and qreject ?
qaccept
4
CONFIGURATION (1) tape contents, (2) current
state, (3) location of read/write head.
11010q700110
5
A TM recognizes a language iff it accepts all and
only those strings in the language.
A language L is called Turing-recognizable or
recursively enumerable iff some TM recognizes L.
A TM decides a language L iff it accepts all
strings in L and rejects all strings not in L.
A language L is called decidable or recursive iff
some TM decides L.
6
A language is called Turing-recognizable or
recursively enumerable (r.e.) if some TM
recognizes it.
A language is called decidable or recursive if
some TM decides it.
7
Theorem If A and ?A are r.e. then A is decidable.
Given a Turing Machine TMA that recognizes A
and a Turing Machine TMR that recognizes ?A, we
can build a new machine that decides A.
How can we prove this?

• Run TMA and TMR in parallel. (Or more
  precisely, interleave them.)
• One of them will eventually recognize the input
  string.
• If TMA recognizes it, then accept.
• If TMR recognizes it, then reject.

8
2n
A TM that decides 0 n 0

• We want to accept iff
• the input string consists entirely of zeros, and
• the number of zeros is a power of 2.

• High-Level Idea.
• Repeatedly divide the number of zeros in half
  until it becomes an odd number.
• If we are left with a single zero, then accept.
• Otherwise, reject.

9
2n
A TM that decides 0 n 0
PSEUDOCODE

1. Sweep from left to right, cross out every other
   0.(Divides number in half.)
2. If in step 1, the tape had only one 0, accept.
3. Else if the tape had an odd number of 0s,
   reject.
4. Move the head back to the first input symbol.
5. Go to step 1.

10
x ? x, L
2n
0 ? 0, L
0 n 0
q2
? ? ?, R
? ? ?, L
x ? x, R
x ? x, R
q0
q1
q3
0 ? ?, R
0 ? x, R
x ? x, R
0 ? 0, R
? ? ?, R
0 ? x, R
? ? ?, R
qaccept
q4
x ? x, R
? ? ?, R
11
q00000
?q1000
?xq300
?x0q40
?x0xq3
?x0q2x
?xq20x
?q2x0x
q2?x0x
12
C aibjck k ij, and i, j, k 1
Example aaabbbbcccccccccccc
3
34 12
4
13
C aibjck k ij, and i, j, k 1
High-Level Idea. For each occurrence of a For
each occurrence of b Delete an occurrence of
c.
14
C aibjck k ij, and i, j, k 1
PSEUDOCODE

1. If the input doesnt match abc, reject.
2. Move the head back to the leftmost symbol.
3. Cross off an a, scan to the right until b. Sweep
   between bs and cs, crossing out one of each
   until all bs are out. If too few cs, reject.
4. Uncross all the bs. If theres another a left,
   then repeat stage 3.If all as are crossed
   out, Check if all cs are crossed off. If yes,
   then accept, else reject.

15
C aibjck k ij, and i, j, k 1
aabbbcccccc
xabbbcccccc
xayyyzzzccc
xabbbzzzccc
xxyyyzzzzzz
16
TURING-MACHINE VARIANTS
Turing machines can be extended in various ways,
but so long as a new TM only reads and writes a
finite number of symbols in each step, an old TM
can still simulate it! Example Turing machines
with multiple tapes. Input comes in on one tape,
and other tapes are used for scratch work.
17
MULTITAPE TURING MACHINES
? Q ? Gk ? Q ? Gk ? L,Rk
18
Theorem Every Multitape Turing Machine can be
transformed into a single tape Turing Machine
0
0
1
.
.
.


0
0
1

19
Theorem Every Multitape Turing Machine can be
transformed into a single tape Turing Machine
0
0
1
.
.
.


0
0
1

20
THE CHURCH-TURING THESIS
Anything that can be computed by algorithm (in
our intuitive sense of the term algorithm) can
be computed by a Turing Machine.
21
We can encode a TM as a string of 0s and 1s
start state
reject state
n states
0n 1 0m 1 0k 1 0s 1 0t 1 0r 1 0u 1
m tape symbols (first k are input symbols)
blank symbol
accept state
( (p, a), (q, b, L) ) 0p10a10q10b10
( (p, a), (q, b, R) ) 0p10a10q10b11
22
Similarly, we can encode DFAs, NFAs, CFGs, etc.
into strings of 0s and 1s
So we can define the following languages
ADFA (B, w) B is a DFA that accepts string
w
ANFA (B, w) B is an NFA that accepts string
w
ACFG (G, w) G is a CFG that generates
string w
23
ADFA (B, w) B is a DFA that accepts string
w
Theorem ADFA is decidable
Proof Idea Simulate B on w
ANFA (B, w) B is an NFA that accepts string
w
Theorem ANFA is decidable
ACFG (G, w) G is a CFG that generates
string w
Theorem ACFG is decidable
Proof Idea Transform G into Chomsky Normal Form.
Try all derivations of length 2w-1
24
UNDECIDABLE PROBLEMS
25
w ? S
w ? S
TM
TM
w ? L ?
w ? L ?
no
no
yes
yes
accept
reject or no output
accept
reject
L is semi-decidable (recursively enumerable,
Turing-recognizable)
L is decidable (recursive)
Theorem L is decidable if both L and ?L are
recursively enumerable
26
There are languages over 0,1 that are not
decidable.
If we believe the Church-Turing Thesis, this is
major it means there are things that formal
computational models inherently cannot do.
We can prove this using a counting argument. We
will show there is no function from the set of
all Turing Machines onto the set of all languages
over 0,1. (Works for any S.)
Then we will prove something stronger There are
semi-decidable (r.e.) languages that are NOT
decidable.
27
Languages over 0,1
Turing Machines
28
Cantors Theorem
Let L be any set and 2L be the power set of L
Theorem There is no map from L onto 2L
Proof
Assume, for a contradiction, that there is an
onto map f L ? 2L
Let S x ? L x ? f(x)
We constructed S so that, for every elem x in L,
the set S differs from f(x) S ? f(x) because x
? S iff x ? f(x)
29
Theorem There is no onto function from the
positive integers to the real numbers in (0, 1)
Proof
Suppose f is such a function
0.28347279
2
1 2 3 4 5
0.88388384
8
0.77635284
6
0.11111111
1
5
0.12345678

1 if nth digit of f(n) ? 1
nth digit of r
0 otherwise
f(n) ? r for all n ( Here, r 0.11101... )
30
Sidenote
Let Z 1,2,3,4. There exists a bijection
between Z and Z ? Z
(or Q)
(1,1) (1,2) (1,3) (1,4) (1,5)
(2,1) (2,2) (2,3) (2,4) (2,5)
(3,1) (3,2) (3,3) (3,4) (3,5)
(4,1) (4,2) (4,3) (4,4) (4,5)
(5,1) (5,2) (5,3) (5,4) (5,5)

.
31
THE MORAL For any set L, 2L always has more
elements than L
32
Not all languages over 0,1 are decidable, in
fact not all languages over 0,1 are
semi-decidable
decidable languages over 0,1
semi-decidable langs over 0,1
Languages over 0,1
Turing Machines
Strings of 0s and 1s
Sets of strings of 0s and 1s
Powerset of L 2L
Set L
33
THE ACCEPTANCE PROBLEM
ATM (M, w) M is a TM that accepts string w
Theorem ATM is semi-decidable (r.e.) but NOT
decidable
ATM is r.e.
Define a TM U as follows On input (M, w), U
runs M on w. If M ever accepts, accept. If M
ever rejects, reject.
U is a universal TM
Therefore, U accepts (M,w) ? M accepts w ? (M,w)
? ATM Therefore, U recognizes ATM
34
ATM (M,w) M is a TM that accepts string w
ATM is undecidable
(proof by contradiction)
Assume machine H decides ATM
H( (M,w) )
Construct a new TM D as follows on input M, run
H on (M,M) and output the opposite of H
D
D
D( M )
Contradiction!
D
D
D
35
OUTPUT OF H

M1
M2
M3
M4
D
M1
accept
accept
M2
reject
accept
M3
accept
reject
M4
accept
reject

?
D
reject
accept
reject
accept
36
Theorem ATM is r.e. but NOT decidable
Theorem ?ATM is not even r.e.!
The Halting Problem is Not Decidable
37
ATM (M,w) M is a TM that accepts string w
ATM is undecidable
A constructive proof
Let machine H semi-decides ATM (Such ? , why?)
H( (M,w) )
Construct a new TM D as follows on input M, run
H on (M,M) and output
D
D
D( M )
D
D
D
D
D
No output
No Contradictions !
H( (D,D) )
38
We have shown Given any presumed machine H for
ATM,we can effectively construct a TM D such
that (D,D) ? ATM but H fails to tell us
that. In other words, For any machine H that
recognizes ATM we can effectively give an
instance where H fails to decide ATM In other
words, Given any good candidate for deciding the
Halting Problem, we can effectively construct an
instance where the machine fails.
39
THE HALTING PROBLEM
HALTTM (M,w) M is a TM that halts on string
w
Theorem HALTTM is undecidable
Proof
Assume, for a contradiction, that TM H
decides HALTTM
We use H to construct a TM D that decides ATM
On input (M,w), D runs H on (M,w)
If H rejects then reject
If H accepts, run M on w until it halts
Accept if M accepts and Reject if M rejects
40
(M,w)
(M,w)
If M halts
D
Does M halt on w?
w
H
M
If M doesnt halt REJECT
41
In many cases, one can show that a language L is
undecidable by showing that if it is decidable,
then so is ATM
We reduce deciding ATM to deciding the language
in question
ATM L
We just showed ATM HaltTM Is HaltTM ATM
?
42
Read chapter 4 of the book for next time