AUBER F16

1
CD5560 FABER Formal Languages, Automata and
Models of Computation Lecture 13 Mälardalen
University 2006
2
ContentIntroduction Universal Turing Machine
Chomsky Hierarchy DecidabilityReducibilityUnco
mputable FunctionsRices TheoremInteractive
Computing, Persistent TMs (Dina Goldin)
3
http//www.turing.org.uk/turing/

• Who was Alan Turing?
• Founder of computer science,
• mathematician,
• philosopher,
• codebreaker,
• visionary man before his time.
• http//www.cs.usfca.edu/www.AlanTuring.net/turing_
  archive/index.html- Jack Copeland and Diane
  Proudfoot
• http//www.turing.org.uk/turing/ The Alan Turing
  Home PageAndrew Hodges

4
Alan Turing

• 1912 (23 June) Birth, London1926-31 Sherborne
  School1930 Death of friend Christopher
  Morcom1931-34 Undergraduate at King's College,
  Cambridge University1932-35 Quantum mechanics,
  probability, logic1935 Elected fellow of King's
  College, Cambridge1936 The Turing machine,
  computability, universal machine1936-38
  Princeton University. Ph.D. Logic, algebra,
  number theory1938-39 Return to Cambridge.
  Introduced to German Enigma cipher
  machine1939-40 The Bombe, machine for Enigma
  decryption1939-42 Breaking of U-boat Enigma,
  saving battle of the Atlantic

5
Alan Turing

• 1943-45 Chief Anglo-American crypto consultant.
  Electronic work.1945 National Physical
  Laboratory, London1946 Computer and software
  design leading the world.1947-48 Programming,
  neural nets, and artificial intelligence1948
  Manchester University1949 First serious
  mathematical use of a computer1950 The Turing
  Test for machine intelligence1951 Elected FRS.
  Non-linear theory of biological growth1952
  Arrested as a homosexual, loss of security
  clearance1953-54 Unfinished work in biology and
  physics1954 (7 June) Death (suicide) by cyanide
  poisoning, Wilmslow, Cheshire.

6
Hilberts Program, 1900

• Hilberts hope was that mathematics would be
  reducible to finding proofs (manipulating the
  strings of symbols) from a fixed system of
  axioms, axioms that everyone could agree were
  true.
• Can all of mathematics be made algorithmic, or
  will there always be new problems that outstrip
  any given algorithm, and so require creative acts
  of mind to solve?

7
TURING MACHINES

8

• Turings "Machines". These machines are humans
  who calculate.
• (Wittgenstein)
• A man provided with paper, pencil, and rubber,
  and subject to strict discipline, is in effect a
  universal machine. (Turing)

9
Standard Turing Machine
Tape
......
......
Read-Write head
Control Unit
10
The Tape
No boundaries -- infinite length
......
......
Read-Write head
The head moves Left or Right
11
......
......
Read-Write head
The head at each time step
1. Reads a symbol 2. Writes a symbol 3. Moves
Left or Right
12
Example
3. Moves Left
13
The Input String
Input string
Blank symbol
......
......
head
Head starts at the leftmost position of the input
string
14
States Transitions
Write
Read
Move Left
Move Right
15
Time 1
......
......
Time 2
......
......
16
Determinism
Turing Machines are deterministic
Not Allowed
Allowed
No lambda transitions allowed in standard TM!
17
Formal Definitions for Turing Machines

18
Transition Function
19
Turing Machine
Input alphabet
Tape alphabet
States
Final states
Transition function
Initial state
blank
20
Universal Turing Machine

21
A limitation of Turing Machines
Better are reprogrammable machines.
22
Solution
Universal Turing Machine
Characteristics

• Reprogrammable machine
• Simulates any other Turing Machine

23
Universal Turing Machine
simulates any other Turing Machine
Input of Universal Turing Machine

• Description of transitions of

• Initial tape contents of

24
Tape 1
Three tapes
Description of
Tape 2
Universal Turing Machine
Tape Contents of
Tape 3
State of
25
Tape 1
Description of
We describe Turing machine as a string of
symbols We encode as a string of symbols
26
(No Transcript)
27
State Encoding
Head Move Encoding
28
Transition Encoding
Transition
Encoding
separator
29
Machine Encoding
Transitions
Encoding
separator
30
Tape 1 contents of Universal Turing Machine
encoding of the simulated machine as
a binary string of 0s and 1s
31
A Turing Machine is described with a binary
string of 0s and 1s.
Therefore
The set of Turing machines forms a language
Each string of the language is the binary
encoding of a Turing Machine.
32
Language of Turing Machines
(Turing Machine 1)
L 010100101, 00100100101111,
111010011110010101,
(Turing Machine 2)

33
The Chomsky Hierarchy

34
The Chomsky Language Hierarchy
Recursively-enumerable
Recursive
Context-sensitive
Context-free
Regular
Non-recursively enumerable
35
(No Transcript)
36
(No Transcript)
37
Theorem
A language is recursively enumerable if and
only if it is generated by an unrestricted
grammar.
38
(No Transcript)
39
is context-sensitive
40
Theorem
A language is context sensitive if and only
if it is accepted by a Linear-Bounded
automaton.
41
Linear Bounded Automata (LBAs) are the same as
Turing Machines with one difference
The input string tape space is the only tape
space allowed to use.
42
(No Transcript)
43
Observation
There is a language which is context-sensitive but
not recursive.
44
Decidability

45
Consider problems with answer YES or NO.
Examples
46
A problem is decidable if some Turing
machine solves (decides) the problem.
Decidable problems
47
The Turing machine that solves a problem answers
YES or NO for each instance.
YES
Input problem instance
Turing Machine
NO
48
The machine that decides a problem

• If the answer is YES
• then halts in a yes state

• If the answer is NO
• then halts in a no state

These states may not be the final states.
49
Turing Machine that decides a problem
YES
NO
YES and NO states are halting states
50
Difference between
Recursive Languages (Acceptera) and Decidable
problems (Avgöra)
For decidable problems
The YES states may not be final states.
51
Some problems are undecidable
There is no Turing Machine that solves all
instances of the problem.
52
A famous undecidable problem
The halting problem
53
The Halting Problem
54
Theorem
The halting problem is undecidable.
Proof
Assume to the contrary that the halting problem
is decidable.
55
YES
halts on
doesnt halt on
NO
56
Input initial tape contents
YES
NO
Encoding of
String
57
(No Transcript)
58
Loop forever
YES
NO
59
Construct machine
Input
(machine )
halts on input
If
Then loop forever
Else halt
60
copy
61
Run machine with input itself
(machine )
Input
halts on input
If
Then loop forever
Else halt
62

on input
If halts then loops forever.
If doesnt halt then it halts.
CONTRADICTION !
63
This means that
The halting problem is undecidable.
END OF PROOF
64
Another proof of the same theorem
If the halting problem was decidable then every
recursively enumerable language would be
recursive.
65
Theorem
The halting problem is undecidable.
Proof
Assume to the contrary that the halting problem
is decidable.
66
There exists Turing Machine that solves the
halting problem.
YES
halts on
doesnt halt on
NO
67
(No Transcript)
68
Turing Machine that accepts and halts on any input
NO
reject
halts on ?
YES
accept
Halts on final state
Run with input
reject
Halts on non-final state
69
Therefore
is recursive.
Since is chosen arbitrarily, we have
proven that every recursively enumerable language
is also recursive.
But there are recursively enumerable languages
which are not recursive.
Contradiction!
70
Therefore, the halting problem is undecidable.
END OF PROOF
71
A simple undecidable problem
The Membership Problem
72
The Membership Problem
Input

• Turing Machine

• String

73
Theorem
The membership problem is undecidable.
Proof
Assume to the contrary that the membership
problem is decidable.
74
There exists a Turing Machine that solves the
membership problem
accepts
YES
NO
rejects
75
We will prove that is also recursive
We will describe a Turing machine that accepts
and halts on any input.
76
Turing Machine that accepts and halts on any input
YES
accept
accepts ?
NO
reject
77
But there are recursively enumerable languages
which are not recursive.
Contradiction!
78
Therefore, the membership problem is undecidable.
END OF PROOF
79
Reducibility

80
(No Transcript)
81
(No Transcript)
82
Example
the halting problem
reduced to
the state-entry problem.
83
The state-entry problem
Inputs
Question
84
Theorem
The state-entry problem is undecidable.
Proof
85
Suppose we have an algorithm (Turing
Machine) that solves the state-entry problem.
We will construct an algorithm that solves the
halting problem.
86
Assume we have the state-entry algorithm
YES
enters
Algorithm for state-entry problem
NO
doesnt enter
87
We want to design the halting algorithm
YES
halts on
Algorithm for Halting problem
NO
doesnt halt on
88
Modify input machine
Single halt state
halting states
89
halts
if and only if
halts on state
90
Algorithm for halting problem
91
Halting problem algorithm
YES
YES
Generate
State-entry
NO
algorithm
NO
92
We reduced the halting problem to the state-entry
problem.
Since the halting problem is undecidable, it must
be that the state-entry problem is also
undecidable.
END OF PROOF
93
Another example
The halting problem
reduced to
the blank-tape halting problem.
94
The blank-tape halting problem
Input
Turing Machine
Question
Does
halt when started with
a blank tape?
95
(No Transcript)
96
Suppose we have an algorithm for the blank-tape
halting problem.
We will construct an algorithm for the halting
problem.
97
Assume we have the blank-tape halting algorithm
halts on blank tape
YES
Algorithm for blank-tape halting problem
NO
doesnt halt on blank tape
98
We want to design the halting algorithm
YES
halts on
Algorithm for halting problem
NO
doesnt halt on
99
step 1
step2
if blank tape
execute
then write
with input
100
halts on input string
if and only if
halts when started with blank tape.
101
Algorithm for halting problem
102
Halting problem algorithm
YES
Blank-tape halting algorithm
YES
Generate
NO
NO
103
We reduced the halting problem to the blank-tape
halting problem.
Since the halting problem is undecidable, the
blank-tape halting problem is also undecidable.
END OF PROOF
104
Summary of Undecidable Problems
105
Blank-tape halting problem
Does machine halt when starting on blank
tape?
State-entry Problem
106
Uncomputable Functions

107
Uncomputable Functions
Domain
Range
A function is uncomputable if it cannot be
computed for all of its domain.
108
Example
An uncomputable function
109
Then the blank-tape halting problem is
decidable.
110
Algorithm for blank-tape halting problem
111
Therefore, the blank-tape halting problem must be
decidable.
However, we know that the blank-tape halting
problem is undecidable.
Contradiction!
112
END OF PROOF
113
Rices Theorem

114
Definition
Non-trivial properties of recursively enumerable
languages
any property possessed by some (not
all) recursively enumerable languages.
115
Some non-trivial properties of recursively
enumerable languages
116
Rices Theorem
Any non-trivial property of a recursively
enumerable language is undecidable.
117
We will prove some non-trivial properties without
using Rices theorem.
118
Theorem
For any recursively enumerable language
it is undecidable whether it is empty.
Proof
We will reduce the membership problem to the
problem of deciding whether is empty.
119
Membership problem Does machine accept
string ?
120
Assume we have the empty language algorithm
YES
empty
Algorithm for empty language problem
NO
not empty
121
We will design the membership algorithm
YES
accepts
Algorithm for membership problem
NO
rejects
122
First construct machine
When enters a final state, compare original
input string with .
123
if and only if
is not empty
124
Algorithm for membership problem
1. Construct
2. Determine if is empty
YES then
NO then
125
Membership algorithm
NO
YES
Check if
construct
YES
is empty
NO
126
We reduced the empty language problem to the
membership problem.
Since the membership problem is undecidable, the
empty language problem is also undecidable.
END OF PROOF
127
Decidabilitycontinued
128
Theorem
Proof
We will reduce the halting problem to the finite
language problem.
129
Assume we have the finite language algorithm
130
We will design the halting problem algorithm
131
Otherwise accept nothing (finite language).
132
(No Transcript)
133
(No Transcript)
134
Machine for halting problem
135
We reduced the finite language problem to the
halting problem.
Since the halting problem is undecidable, the
finite language problem is also undecidable.
END OF PROOF
136
Theorem
Proof
We will reduce the halting problem to the two
strings of equal length- problem.
137
Assume we have the two-strings algorithm
138
We will design the halting problem algorithm
139
Initially, simulates on input .
When enters a halt state, accept symbols
or .
(two equal length strings)
140
(No Transcript)
141
Algorithm for halting problem
142
Machine for halting problem
YES
YES
Check if
construct
NO
NO
has two equal length strings
143
We reduced the two strings of equal length -
problem to the halting problem.
Since the halting problem is undecidable, the two
strings of equal length problem is also
undecidable.
END OF PROOF
144
Rices sats
Om ? är en mängd av Turing-accepterbara språk som
innehåller något men inte alla sådana språk, så
kan ingen TM avgöra för ett godtyckligt
Turing-accepterbart språk L om L tillhör ? eller
ej.
145
Exempel
Givet en Turingmaskin M, kan man avgöra om alla
strängar som accepteras av M börjar och slutar på
samma tecken?
146
Oavgörbart
Problemet handlar om en icke-trivial
språkegenskap. Det finns TMer vars accepterade
strängar har egenskapen i fråga, och det finns
TMer vars accepterade strängar inte har
egenskapen.
147
Formellt

• ? L TM accepterbara språk vars strängar
  börjar och slutar på samma tecken.

148
 Interaction Conjectures, Results, and Myths

• Dina GoldinUniv. of Connecticut, Brown
  Universityhttp//www.cse.uconn.edu/dqg

149
Fundamental Questions Underlying Theory of
Computation

• What is computation?
• How do we model it?

150
Shared Wisdom(from our undergraduate Theory of
Computation courses)

• computation finite transformation of input to
  output
• input finite size (e.g. string or number)
• closed system all input available at start, all
  output generated at end
• behavior functions, transformation of input data
  to output data
• Church-Turing thesis Turing Machines capture
  this (algorithmic) notion of computation

Mathematical worldview All computable problems
are function-based.
151
The Mathematical Worldview

• The theory of computability and
  non-computability is usually referred to as the
  theory of recursive functions... the notion of TM
  has been made central in the development."
• Martin Davis, Computability Unsolvability,
  1958
• Of all undergraduate CS subjects, theoretical
  computer science has changed the least over the
  decades.
• SIGACT News, March 2004
• A TM can do anything that a computer can do.
• Michael Sipser, Introduction to the Theory of
  Computation, 1997

152
The Operating System Conundrum
Real programs, such as operating systems and word
processors, often receive an unbounded amount of
input over time, and never "finish" their task.
Turing machines do not model such ongoing
computation well TM entry, Wikipedia
If a computation does not terminate, its
useless but arent OSs useful??
153
Rethinking Shared Wisdom(what do computers do?)

• computation ongoing process which performs a
  task or delivers a service
• dynamically generated stream of input tokens
  (requests, percepts, messages)
• open system later inputs depend on earlier
  outputs and vice versa (I/O entanglement, history
  dependence)
• behavior processes, components, control devices,
  reactive systems, intelligent agents
• Wegners conjecture Interaction is more powerful
  than algorithms

• computation finite transformation of input to
  output
• input finite-size (string or number)
• closed system all input available at start,
  all output generated at end
• behavior functions, algorithmic transformation
  of input data to output data
• Church-Turing thesis Turing Machines capture
  this (algorithmic) notion of computation

154
Example Driving home from work
Algorithmic input a description of the world (a
static map) Output a sequence of pairs of s
(time-series data)- for turning the wheel-
for pressing gas/break Similar to classic AI
search/planning problems.
155
Driving home from work (cont.)
But in a real-world environment, the output
depends on every grain of sand in the road
(chaotic behavior). Can we possibly have a map
thats detailed enough?
Worse yet the domain is dynamic. The output
depends on weather conditions, and on other
drivers and pedestrians. We cant possibly be
expected to predict that in advance!
Nevertheless the problem is solvable! Google
autonomous vehicle research
156
Driving home from work (cont.)
The problem is solvable interactively. Interactiv
e input stream of video camera images, gathered
as we are driving
Output the desired time-series data, generated
as we are driving similar to control systems,
or online computation A paradigm shift in the
conceptualization of computational problem
solving.
157
Outline

• Rethinking the mathematical worldview
• Persistent Turing Machines (PTMs)
• PTM expressiveness
• Sequential Interaction
• Sequential Interaction Thesis
• The Myth of the Church-Turing Thesis
• the origins of the myth
• Conclusions and future work

158
Sequential Interaction

• Sequential interactive computation
• system continuously interacts with its
  environment by alternately accepting an input
  string and computing a corresponding output
  string.
• Examples
• method invocations of an object instance in an
  OO language
• a C function with static variables
• queries/updates to single-user databases
• recurrent neural networks

- control systems - online computation -
transducers - dynamic algorithms - embedded
systems
159
Sequential Interaction Thesis

• Universal PTM simulates any other PTM
• Need additional input describing the PTM (only
  once)
• Example simulating Answering Machine
• (simulate AM, will-do),
• (record hello, ok), (erase, done), (record John,
  ok),
• (record Hopkins, ok), (playback, John Hopkins),
• Simulation of other sequential interactive
  systems is analogous.

160
Church-Turing Thesis Revisited

• Church-Turing Thesis
• Whenever there is an effective method for
  obtaining the values of a mathematical function,
  the function can be computed by a Turing Machine
• Common Reinterpretation (Strong Church-Turing
  Thesis)
• A TM can do (compute) anything that a
  computer can do

• The equivalence of the two is a myth
• the function-based behavior of algorithms does
  not capture all forms of computation
• this myth has been dogmatically accepted by the
  CS community
• Turing himself would have denied it
• in the same paper where he introduced what we now
  call Turing Machines, he also introduced choice
  machines, as a distinct model of computation
• choice machines extend Turing Machines to
  interaction by allowing a human operator to make
  choices during the computation.

161
Origins of the Church-Turing Thesis Myth

• A TM can do anything that a computer can do.
• Based on several claims
• A problem is solvable if there exists a Turing
  Machine for computing it.
• A problem is solvable if it can be specified by
  an algorithm.
• Algorithms are what computers do.
• Each claim is correct in isolation
• provided we understand the underlying assumptions
• Together, they induce an incorrect conclusion
• TMs solvable problems algorithms computation

162
Deconstructing the Turing Thesis Myth (1)

• TMs solvable problems
• Assumes
• All computable problems are function-based.
• Reasons
• Theory of Computation started as a field of
  mathematics mathematical principles were
  adopted for the fundamental notions of
  computation, identifying computability with the
  computation of functions, as well as with Turing
  Machines.
• The batch-based modus operandi of original
  computers did not lend itself to other
  conceptualizations of computation.

163
Deconstructing the Turing Thesis Myth (2)

• solvable problems algorithms
• Assumes
• Algorithmic computation is also function based
  i.e., the computational role of an algorithm is
  to transform input data to output data.
• Reasons
• Original (mathematical) meaning of algorithms
• E.g. Euclids greatest common divisor algorithm
• Original (Knuthian) meaning of algorithms
• An algorithm has zero or more inputs, i.e.,
  quantities which are given to it initially
  before the algorithm begins. Knuth68

164
Deconstructing the Turing Thesis Myth (3)

• algorithms computation
• Reasons
• The ACM Curriculum (1968) Adopted algorithms as
  the central concept of CS without explicit
  agreement on the meaning of this term.
• Textbooks When defining algorithms, the
  assumption of their closed function-based nature
  was often left implicit, if not forgotten
• An algorithm is a recipe, a set of instructions
  or the specifications of a process for doing
  something. That something is usually solving a
  problem of some sort. RiceRice69
• An algorithm is a collection of simple
  instructions for carrying out some task.
  Commonplace in everyday life, algorithms
  sometimes are called procedures or recipes...
  Sipser97

165
Outline

• Rethinking the mathematical worldview
• Persistent Turing Machines (PTMs)
• PTM expressiveness
• Sequential Interaction
• The Myth of the Church-Turing Thesis
• Conclusions and future work

166
The Shift to Interaction in CS
Algorithmic
Interactive
Computation transforming input to output Computation carrying out a task over time
Logic and search in AI Intelligent agents, partially observable environments, learning
Procedure-oriented programming Object-oriented programming
Closed systems Open systems
Compositional behavior Emergent behavior
Rule-based reasoning Simulation, control, semi-Markov processes
167
The Interactive Turing Test

• From answering questions to holding discussions.
• Learning from -- and adapting to -- the
  questioner.
• Book intelligence vs. street smarts.
• It is hard to draw the line at what is
  intelligence and what is environmental
  interaction. In a sense, it does not really
  matter which is which, as all intelligent systems
  must be situated in some world or other if they
  are to be useful entities. Brooks

168
Modeling Interactive Computation PTMs in
Perspective

• Many other interactive models
• Reactive MP and embedded systems
• Dataflow, I/O automata Lynch, synchronous
  languages, finite/pushdown automata over infinite
  words
• Interaction games Abramsky, online algorithms
  Albers
• TM extensions on-line Turing machines Fischer,
  interactive Turing machines Goldreich...
• Concurrency Theory
• Focuses on communication (between concurrent
  agents/processes) rather than computation
  Milner
• Orthogonal to the theory of computation and TMs.
• What makes PTMs unique?
• Provably more expressive than TMs.
• Bridging the gap between concurrency theory
  (labeled transition systems) and traditional TOC.

169
Future Work 3 conjectures

• Theory of Sequential Interaction
• conjecture notions analogous to computational
  complexity, logic, and recursive functions can be
  developed for sequential interaction computation
• Multi-stream interaction
• From hidden variables to hidden interfaces
• conjecture multi-stream interaction is more
  powerful than sequential interaction Wegner97
• Formalizing indirect interaction
• Interaction via persistent, observable changes to
  the common environment
• In contrast to direct interaction (via message
  passing)
• conjecture direct interaction does not
  capture all forms of multi-agent behaviors

170
Referenceshttp//www.cse.uconn.edu/dqg/papers/
Wegner97 Peter WegnerWhy Interaction is more
Powerful than AlgorithmsCommunications of the
ACM, May 1997 EGW04 Eugene Eberbach, Dina
Goldin, Peter Wegner Turing's Ideas and Models
of Computationbook chapter, in Alan Turing Life
and Legacy of a Great Thinker, Springer
2004 IC04 Dina Goldin, Scott Smolka, Paul
Attie, Elaine SondereggerTuring Machines,
Transition Systems, and InteractionInformation
Computation Journal, 2004 GW04 Dina Goldin,
Peter WegnerThe Church-Turing Thesis Breaking
the Mythpresented at CiE 2005, Amsterdam, June
2005 to be published in LNCS