CSC 3130: Automata theory and formal languages

1
Fall 2008
The Chinese University of Hong Kong
CSC 3130 Automata theory and formal languages
Undecidable Languages
Andrej Bogdanov http//www.cse.cuhk.edu.hk/andrej
b/csc3130
2
Problems about automata

• When we talked about DFA minimization, we asked
  questions like

Is this DFA minimal?
q1
1
q2
1
1
Answer Yes, because
e
e
e
q3
q0
q1
q2
3
Problems about automata

• Can we make a TM that answers questions like

Is this DFA minimal?
We can implement the DFA minimization algorithm
By the C-T thesis, a Turing Machine can do this
4
Problems about automata

• What does a TM that answerslook like?

Is this DFA minimal?
A DFA is (Q, S, d, q0, F)
M
q0,q1,q2,q30,1(q0,0,q0)(q0,1,q1)(q1,0,q0)...
5
Taking an automaton as an input
M
q0,q1,q2,q3,0,1,(q0,0,q0),(q0,1,q1),...

• This is a TM that takes the description ltDgt of a
  DFA D as an input and decides the language

MINDFA ltDgt D is a minimal DFA
6
Acceptance problems about automata
ADFA ltD, wgt D is a DFA that accepts input w
ANFA ltN, wgt N is an NFA that accepts w
AREX ltR, wgt R is a regular expression that
generates w
Which of these is decidable?
7
Acceptance problems about automata
decidable
ADFA ltD, wgt D is a DFA that accepts input w

• The following TM decides ADFA

M
On input ltD, wgt, where D is a DFA and w is a
string Simulate D on input w If the
simulation accepts, accept. If it doesnt,
reject.
8
Acceptance problems about automata
decidable
ANFA ltN, wgt N is an NFA that accepts input w

• The following TM decides ANFA

M
On input ltN, wgt, where N is an NFA and w is a
string Convert N to a DFA D using
the subset construction Run the TM for ADFA
on input ltD, wgt
9
Other problems about automata
decidable
MINDFA ltDgt D is a minimal DFA
On input ltDgt, where D is a DFA Run the DFA
minimization algorithm on D to obtain a
DFA D If D D accept, otherwise reject.
decidable
EDFA ltDgt D accepts no inputs
10
Problems about context-free grammars
decidable
ACFG ltG, wgt G is a CFG that generates w
On input ltG, wgt, where G is a CFG and w is a
string Convert G to Chomsky Normal Form
Run the Cocke-Younger-Kasami algorithm If the
top entry contains the start variable, accept
Otherwise reject
11
Equivalence
decidable
EQREX ltR1, R2gt R1 and R2 are regular
expressions that generate the same strings
On input ltR1, R2gt, where R1, R2 are reg
expressions Convert both R1 and R2 to DFAs D1
and D2 Run the DFA minimization algorithm on
both to obtain minimized DFAs D1 and
D2 If D1 D2 accept, otherwise reject
EQCFG ltG1, G2gt G1 and G2 are context-free
grammars that generate the same strings
undecidable
12
Are Turing Machines computers?
computer
input data
output data
instructions (program)
Can a Turing Machine take instructionsas part of
its input?
13
The Universal Turing Machine
U
input x
M on input x
program ltMgt
The universal TM U takes as inputs a program
Mand a string x and simulates M on x
The program M itself is specified as a TM!
14
Turing Machines as strings
M
?/?R
qa
0/0R

• A Turing Machine is (Q, ?, ?, ?, q0, qacc, qrej)

q
1/1R
qr

• This Turing Machine can be described by the string

ltMgt q,qa,qr,0,1,0,1,?, (q,q,?/?R)
(q,qa,0/0R) (q,qr,1/1R), q,qa,qr
15
The universal Turing Machine
U
program ltMgt
q,qa,qr0,10,1,?(q,q?/?R),(
input x for M
001
state
q
U on input ltM, xgt will simulate M on input x
16
Acceptance of Turing Machines
ATM ltM, wgt M is a TM that accepts w

• Run the universal TM U on input ltM, wgt
• Simulation is correct
• If M accepts w, we accept
• If M rejects w, we reject
• If M loops on w, we loop
• This TM recognizes but does not decide ATM

17
Recognizing versus deciding
qacc
qrej
accept
reject
loop
halt
The language recognized by a TM is the set ofall
inputs that make it accept
A TM decides language L if it recognizes L
andhalts (does not loop) on every input
18
Undecidability

• Turings Theorem
• Before we prove this, lets observe one thing

The language ATM is undecidable.
A Turing Machine M can be given its
own description ltMgt as an input!
19
Idea of proof
If some TM decides ATM, then this TM is so
powerful that it will destroy itself.
20
Proof of undecidability

• Proof by contradiction

Suppose ATM is decidable.
Then there is a TM H that decides ATM
What happens when w ltMgt?
21
Proof of undecidability
accept if M accepts ltMgt
H
ltM, ltMgtgt
reject if M rejects ltMgt or M loops on ltMgt
Let H be a TM that does the opposite of H
H
22
Proof of undecidability
accept if M accepts ltMgt
H
ltM, ltMgtgt
reject if M rejects ltMgt or M loops on ltMgt
if M rejects ltMgt or M loops on ltMgt
accept
H
ltM, ltMgtgt
reject if M accepts ltMgt
23
Proof of undecidability
if M rejects ltMgt or M loops on ltMgt
accept
H
ltM, ltMgtgt
reject if M accepts ltMgt
Let D be the following TM
copy
24
Proof of undecidability
if M rejects ltMgt or M loops on ltMgt
accept
D
ltMgt
reject if M accepts ltMgt
What happens when M D?
If D accepts ltDgt
then D rejects ltDgt
H never loops, so H and D never loop either
so D does not exist!
If D rejects ltDgt
then D accepts ltDgt
25
Proof of undecidability conclusion

• Proof by contradiction
• We assumed ATM was decidable
• Then we had Turing Machines H, H, D
• But D does not exist!
• Conclusion

The language ATM is undecidable.
26
What happened?
input w

00
e
0
1
M1
acc
rej
rej
acc
M2

rej
acc
loop
rej
Turing Machine
M3
rej
loop
rej
rej


We can write an infinite table for every pair (M,
w)
27
What happened?

ltM4gt
ltM1gt
ltM2gt
ltM3gt
M1
acc
loop
rej
acc
M2

rej
acc
rej
acc
M3
loop
rej
loop
rej


Now lets look only at those w that describe a TM
M
28
What happened?

ltM4gt
ltM1gt
ltM2gt
ltM3gt
M1
acc
loop
rej
acc
M2

rej
acc
rej
acc


D
rej
rej
acc
rej


If ATM is decidable, then TM D is in the table
29
What happened?

ltM4gt
ltM1gt
ltM2gt
ltM3gt
M1
acc
loop
rej
acc
M2

rej
acc
rej
acc


D
rej
rej
acc
rej
D does the opposite of the diagonal entries
ltMgt
accept if M rejects or loops on ltMgt
D
reject if M accepts ltMgt
30
What happened?

ltM4gt
ltM1gt
ltM2gt
ltM3gt
ltDgt
M1
acc
loop
rej
acc
loop
M2

rej
acc
rej
acc
acc


?

D
rej
rej
acc
rej
We run into trouble when we look at (D, ltDgt)!
31
Unrecognizable languages
The language ATM is recognizable but not
decidable.

• How about languages that are not recognizable?

The language ATM is not recognizable.
32
Unrecognizable languages

• Theorem
• We know ATM is recognizable, so if ATM were
  also,then ATM would be decidable
• Impossible by Turings Theorem

If L and L are both recognizable, then L
isdecidable.
33
Unrecognizable languages
If L and L are both recognizable, then L
isdecidable.

• Proof idea

34
Unrecognizable languages
If L and L are both recognizable, then L
isdecidable.

• Proof attempt

Problem If M loops on w, we will never get to
step 2!
Let M TM for L, M TM for L
On input w,
Simulate M on input w. If it accepts, accept.
?
Simulate M on input w. If it accepts, reject.
?
35
Bounded simulation
If L and L are both recognizable, then L
isdecidable.

• Decider for L

Let M TM for L, M TM for L
On input w,
For k 0 to infinity
Simulate M on input w for k steps. If it accepts,
accept.
Simulate M on input w for k steps. If it
accepts, reject.
36
Another undecidable language
HALTTM ltM, wgt M is a TM that halts on input
w
HALTTM is an undecidable language

• To prove this, we will show that so by
  Turings Theorem it is undecidable.

If HALTTM can be decided, so can ATM.
37
Undecidability of halting
If HALTTM can be decided, so can ATM.

• Suppose R decides HALTTM

accept if M halts on w
R
ltM, wgt
reject if M loops on w

• We want to use R to design D for ATM

accept if M accepts w
?
ltM, wgt
reject if M rejects or loops on w
38
Undecidability of halting
accept if M halts on w
R
ltM, wgt
reject if M loops on w
D
accept if M accepts w
ltM, wgt
reject if M rejects or loops on w
39
More undecidable problems
L1 ltMgt M is a TM that accepts input e
L2 ltMgt M is a TM that accepts some input
L3 ltMgt M is a TM that accepts all inputs
L4 ltM, Mgt M and M accept the same inputs
Next lecture...
40
Gödels incompleteness theorem
Some mathematical statementsare true but not
provable.

• Turings proof of Gödels Theorem

If every true statement is provable, then ATM can
be decided.
41
Proof sketch of Gödels theorem

• Suppose every true statement has a proof
• For every TM M and input w, one of the
  statementsis true, so it must have a proof

M accepts w
M does not accept w
or
42
Proof sketch of Gödels theorem

• Consider this TM D for ATM

On input ltM, wgt For k 0 to infinity For all
possible proofs p of length k If p is a proof
of M accepts w, accept. If p is a proof of
M does not accept w, reject.
One of them is true, so it has a proof p
Eventually, D will find this proof