Notes on TMs

1
Notes on TMs
2
Functions and Turing Machines

• TMs can compute a function.
• A TM takes as input a string and produces as
  output another string.
• Output?? -- the string that is left on the tape
  when (if) the TM halts.
• So any TM can be thought of as a (possibly
  partial) f function from ? to ?.
• Up to now, TMs have simply been used as language
  acceptors the contents on the tape were
  unimportant, all that mattered was whether or not
  a string was accepted. Now that we are
  interested in functions, the contents of the
  string are important

3

• Def Let T(Q,?,?,q0,?) be a TM and let f be a
  partial function mapping ???. We say that T
  computes f if for every x?? for which f(x) is
  defined, we have
• (q0,?x)- (ha,?f(x)) if f(x) is not defined,
  then x is not accepted by T.
• To slightly generalize this, suppose f is a
  partial function mapping (?)k?? for some k?N.
  We say T computes f if for every k-tuple (x1, x2,
  , xk)x, if f(x) is defined then
  (q0,?x1?x2?xk)- (ha,?f(x)) and if f(x) is not
  defined then x is not accepted by T.
• Say f is Turing computable if there is a TM that
  computes f

4

• Deal with numerical functions by choosing a way
  to represent numbers as strings. We will
  restrict ourselves to the natural numbers, and
  use unary notation The number n is represented
  as the string of n 1s
• 0?10 and n1?1n1
• Def Let T(Q,1,?,q0,?) be a TM and let f be a
  (possibly partial) function mapping N?N. We say
  T computes f if for every n?N, if f(n) exists
  then
• (q0, ?1n)?(ha, ?1f(n)), and if f(n) doesnt
  exist then T does not accept input 1n.
• Note computing means ending up in ha
• The definition is extended to Nk?N exactly the
  same way as we did in the previous definition.

5
TM computing f(n) n(mod 2)
Ex We draw a TM computing f(n) n(mod 2)
6
Characteristic Functions

• Consider a language L??. Closely associated to
  L is its characteristic function ?L??0,1
  defined by

7

• When L is accepted by a TM T, the TM reached ha
  only when the strings in L are input to T. In
  computing ?L, (if it can be computed!!) we now
  accept every string, and distinguish the strings
  with the output.
• If we have a TM computing ?L, it is easy to
  obtain a TM accepting L modify T so that
  whenever an output of 0 is left, the machine
  enters hr instead of ha.
• However, given a TM accepting L, it is NOT
  trivial to construct a TM computing ?L. This is
  because of infinite loops there may be some
  strings not in L that cause T to enter an
  infinite loop, and its not obvious how to modify
  a TM to halt and output 0 for these cases.
• This question will be the subject of the next
  section.

8
Combining Turing Machines

• We have hinted that TMs are very powerful and
  general machines. Much of the work that occurs
  during the execution of a TM involves moving from
  one end of a string to another, and similar
  monotonous tasks.
• For more complicated algorithms/programs, it is
  not practical to show all the details.
• The natural way to deal with more complicated
  algorithms or programming assignments build
  simple and reusable components and construct
  composite TMs by executing one then another.

9
T1T2

• Let T1 and T2 be TMs with disjoint sets of states
  and transition functions ?1 and ?2. We will
  denote the composite TM by T1T2.
• The states of T1T2 is the union of the states of
  T1 and T2. T1T2 begins in the initial state of T1
  and executes the moves of T1 (using ?1) up until
  the point where T1 would halt. For any move that
  would cause T1to halt in ha, T1T2 executes the
  same move (ie replacing symbols and moving
  tapehead) but instead moves to the initial state
  of T2. From this point on, the moves of T1T2 use
  the function ?2.
• Note if T1 orvT2 would reject during this
  process, then so will T1T2. Also, T1T2 accepts
  iff T2 accepts.
• In a transition diagram, we would write T1? T2
  to represent the composite T1T2
• Write T1 b? T2 to mean same as above except For
  any move that causes T1 to halt in ha if the tape
  symbol is b it executes T2.

10
Basic TMs

• Examples from text to
• Copy a string
• Delete a symbol
• Reverse a string

11
Variations on TMs

• We could allow the tape to be infinite in both
  directions. We could restrict the moves of
  tapehead to L,R, rather than L,R,S. We could
  restrict moves to include writing on the tapehead
  or moving the tapehead but not both.
• We could also allow the possibility of having
  multiple tapes. Additional tapes can make it
  easier to construct a TM that executes a desired
  algorithm so are very useful as some algorithms
  are much easier to implement on multitape TMs.
• There turns out to be no difference in the
  ultimate computing power of a TM when extra
  tapes are added. There is also no additional
  computing power when the tape is infinite in both
  directions or when moves are restricted in other
  ways as in the first bullet above.
• By ultimate computing power, we arent talking
  about speed or efficiency or memory requirements
  we are simply concerned with whether or not the
  problem can be solved. That is machines of type
  B are as powerful as machines of type A if
  whenever there is a machine TA of type A there is
  a machine TB of type B that accepts exactly the
  same inputs strings as TA and produces exactly
  the same output as TA whenever it halts in the
  accepting state.
• We focus our attention on multitape TMs

12

• What do we mean by additional tapes? Lets say
  we wanted 2 tapes then there would be 2
  tapeheads, one for each tape. And the transition
  function would depend on the state, as well as
  the symbol on each tapehead.
• A n-tape TM is a 5-tuple T(Q,?,?,q0,?) which is
  the same as a normal TM except for the transition
  function which now takes the form
• ?Q?(???)n?(Q?ha,hr)?(???)n?R,L,S
  n
• A configuration of a n-tape TM is a (n1)-tuple
  of the form
• (q, x1a1y1, x2a2y2, , xnanyn) the initial
  configuration of an n-tape TM on input x is
  (q0, ?x, ?, , ?).
• The first tape is where the input is put. We
  also call the output of a n-tape TM to be the
  final contents of tape 1 when it halts. Tapes 2
  through n are used simply for auxiliary working
  space. In particular if a TM computes the
  function f, it halts in some configuration (ha,
  ?f(x),)

13
1-Tape is Enough

• It is easy to see that for all n?2, a n-tape TM
  is at least at powerful as an ordinary TM. To
  simulate a 1 tape TM, a TM with n tapes simply
  acts as if tape 1 were its only one and leaves
  the others blank.We now show the converse.
• Theorem Let n?2 and let T1(Q1,?,?1,q1,?1) be a
  n-tape TM. Then there is a 1-tape TM
  T2(Q2,?,?2,q2,?2) with ?1??2 satisfying the
  following two conditions
• (1) L(T1)L(T2) ie -- T1 accepts x iff T2
  accepts x.
• (2) If (q1, ?x, ?, , ?)- T1(ha, yaz, y2a2z2,
  , ynanzn) then (q2, ?x)- T2(ha, yaz)
• (ie if T1 accepts x, then T2 accepts x and
  produces the same output as T1).

14

• Proof I give the proof for n2 it easily
  generalizes to an arbitrary n. We construct a
  one-tape TM that simulates a two-tape TM. How do
  we do this? By changing what we consider a
  tape. We make a single tape look like two
  tapes by using a more complicated tape alphabet.
  More specifically, a single square in our new TM
  will essentially be a pair of symbols, each
  symbol representing a square from tape1 and a
  square from tape2. By adding even more symbols,
  we can take care of how to deal with the fact
  that T1 has 2 tapeheads.
• We describe in detail the elements that ?2
  contains.
• (1) ?2 contains every symbol in ?1?? reqd
  to code the inputs and outputs of T2 .
• (2) ?2 contains every element of ???. As
  suggested above, a symbol (X,Y)???? on tape
  square i represents X on square i for first tape
  and Y for square i for second tape. T2 can be
  thought of as having two tracks, each track
  corresponding to a tape of T1. These two tracks
  will not initially exist, but will rather but
  created as T2 moves its tape head further to the
  right.
• (3) Let ?a a??. (? contains the same
  symbols as ? except we mark them all with a to
  distinguish them from elements of ?). Then ?2
  contains elements of (???)?(???)?(???)
• At every point during the simulation, there
  will be one primed symbol on each track to
  designate the location of the tapehead on the
  corresponding tape of T1 ( a pair with 2 primed
  symbols indicates that the tapehead locations in
  T1 are the same).

15

• For example, given the following tape sequence
  for T2(X,Y) (Z,?) (X,Y) (?,?) .. this
  sequence represents the following tape sequences
  for T1
• Tape1 XZX?
• Tape2 Y?Y?
• (4) The extra symbol . It is inserted initially
  into the leftmost square of T2, making it easy to
  find the beginning of the tape whenever we want.
• After T2 inserts the symbol , it changes the
  blank in square 1 to (?,?), signifying that the
  head on each track starts off on their first
  square. And then move the actual head back to
  square 0. From now on the two tracks will extend
  as far as T2 has ever moved its tapehead to the
  right whenever it advances one square further
  it converts from the old single symbol to the new
  double symbol.
• Now the actual simulation starts, that is we
  want T2 simulate a move of T1. Suppose for T1
  the move is of the form ?(p,a1,a2)(q,b1,b2,D1,D2)
  .
• T2 must determine which move to make by locating
  the primed symbols on the two tracks it then
  carries out the move by making the appropriate
  changes to both its tracks, including the
  positions of the tape heads of T1. Further
  details of the steps T2 makes to simulate a
  single move of T1 are detailed explicitly in the
  text.
• Corollary Any language that is accepted by an
  n-tape TM can be accepted by an ordinary Any
  function that is computed by an n-tape TM can be
  computed by an ordinary TM

16
Nondeterministic TMs

• It turns out that adding nondeterminism to TMs
  does not affect what problems can be solved,
  though it does lead to some of the most important
  problems in computer science (P vs NP).
• A nondeterministic turing machine (NTM) is
  defined exactly the same as a TM except that
  values of the transition function ? are subsets
  rather than elements of the set
  (Q?ha,hr)?(???)?R,L,S
• We no longer need to say that ? is a partial
  function because now the machine can crash by
  having ?(q,a)?.
• The notation stays the same for configurations
• (p, xay)- (q,wbz) means that at least one
  sequence of zero or more moves takes T from the
  left configuration to the right.

17

• The idea of output will no longer be as useful
  as in the deterministic case because for a given
  NTM there could conceivably be infinitely many
  outputs. When we compare NTMs to TMs, we
  restrict ourselves to machines used as language
  acceptors.
• Every TM can obviously be considered a NTM. The
  converse is also true!
• Theorem Let T1(Q1,?,?1,q1,?1) be a NTM. Then
  there exists a TM T2(Q2,?,?2,q2,?2) such that
  L(T1)L(T2).

18

• Proof I only very briefly sketch this proof. We
  want T2 to have the property that x?L(T2) iff
  there is some sequence of moves of T1 on input x
  that causes it to accept.
• The strategy for constructing T2 is simply to let
  is try every sequence of moves of T1, one
  sequence at a time, accepting iff it finds a
  sequence that would cause T1 to halt in the
  accepting state.
• There may be many configurations for which T1 has
  a choice of moves, but there is a fixed upper
  bound on the number of moves it might ever have
  we assume this number is 2, for a bigger number
  the argument we give will easily generalize.
• We further assume that whenever there are any
  moves at all, there are exactly two which we
  label 0 and 1. The order is arbitrary, and it
  doesnt matter if both are the same move.
• We use a computation tree to represent the
  sequence of moves T1 might make on input x. T2s
  job is thus searching the tree for accepting
  configurations. The tree could be infinite, so
  we use a breadth-first approach T2 tries all
  single moves, then all sequences of 2 moves, then
  all sequences of 3 moves, etc etc. This is
  inefficient, but we dont care!
• We take advantage of the previous theorem by
  giving T2 3 tapes. The first tape stores the
  input string, and it never changes. The second
  tape keeps track of the sequence of moves of T1
  that T2 is trying to execute. The 3rd is the
  working tape corresponding to T1s tape where T2
  carries out the steps specified by the current
  string on tape 2. Every time T2 begins trying a
  new sequence, the third tape is erased and the
  input from tape 1 is recopied onto it.

19

• Many details are discussed in text see Figure
  9.16 detailing the simulation of a NTM by a 3
  tape TM using 5 components
• InitializeTapes23, CopyInput, Execute,
  EraseTape3 NextSequence,
• looping through components 2nd, 3rd,4th, 5th,
  2nd, etc.
• Details of the component TMs required are given
  in that section or in earlier sections.

20
Universal Turing Machines

• Thus far, we have given the impression that is a
  turing machine is constructed to execute one
  specific algorithm. If a TM computes a function
  f(x), then to compute g(x) we (probably) need a
  totally different TM.
• However, this is not how we use computers today
  modern computers are flexible in that the tasks
  they perform is to execute the instructions
  stored in its memory.
• Electronic computers were originally very limited
  in what they could do, and changing their job
  meant rewiring the machine. However, in 1936
  Turing published a paper that changed all this.
  Turing was probably the first to anticipate our
  modern day computers that are more flexible.
• He described a universal computing machine that
  works as follows it is a TM TU whose input
  consists essentially of a program and a data set
  for the program to process. We input into TU a
  turing machine T and a string x, and TU
  calculates the effect of running x through
  machine T. T and x are encoded so that they can
  be used as input for TU.

21

• The first step is to establish a system for
  encoding strings and turing machines. We want to
  encode a turing machine T as a string e(T) and a
  string x as another string e(x). e must not
  destroy any information given e(T), we must be
  able to reconstruct T. Our universal turing
  machine is going to have alphabet 0,1, while T
  may have a much larger alphabet. We will assign
  a positive interger to each state, tape symbol,
  and the 3 directions S,L, R. We will then
  define a way to encode a move.
• A slight technical problem we want the encoding
  function to be 1-1. For example, two TMs with
  the same rules but with different state names are
  essentially the same and should have the same
  encoding. How do we fix this?
• Convention Fix two infinite sets ?q1,q2,q3,
  and ?a1,a2,q3,. Assume that for any turing
  machine T(Q,?,?,q0,?) that Q?? and ???.
• Note this is not a restriction at all the names
  assigned to a state are arbitrary, and we just
  need to give ? all the symbols we could ever
  possibly want. (For example, there is a fixed
  set of 256 ASCII characters.)

22
Encoding Function)

• First, associate to each tape symbol (including
  ?), each state (including ha, hr) and each of the
  directions, a string of 0s. Let
• s(?)0
• s(ai)0i1 (ai??)
• s(ha)0
• s(hr)00
• s(qi)0i2
• s(S)0
• s(L)00
• s(R)000
• Each move m of a TM is described by a formula m
  ?(p,a)(q,b,D).
• We encode m by the formula
• e(m)s(p)1s(a)1s(q)1s(b)1s(D)1
• And for any TM T with initial state q, T is
  encoded as
• e(T)s(q)1e(m1)1e(m2)1e(mk)
  1
• where m1, , mk
  are the distinct moves of T, arranged in
  arbitrary order.

23

• The input to TU will consist of a string of the
  form e(T)e(z).
• Want TU to accept e(T)e(z) iff T accepts z,
• Want TU to output the encoded output from T
  running z. We briefly describe how to construct
  TU.
• TU will be have 3 tapes
• The first tape will of course be the input and
  output tape. It will initially contain the
  string e(T)e(z).
• The second tape will be the working tape that
  simulates string z running on T.
• The third tape will contain the encoded state
  that T is in.
• TU first moves the string e(z) (except for its
  initial 1) from the end of tape 1 to tape 2,
  beginning in the 3rd square from the left. (T
  begins with its leftmost square blank, so TU will
  write 01 in squares 1 and 2 of tape 2. Note
  s(?)0.) TU next copies the encoded form of Ts
  initial state from the beginning of tape 1 onto
  tape 3, beginning in square 1, and deletes it
  from tape 1.

24

• After these initial steps, TU is ready to begin
  simulating the action of T, which is encoded on
  tape 1, on the string, which is encoded on tape
  2.
• As the simulation starts, the 3 tape heads are
  all in square 1.
• The next move of T at any point is determined by
  Ts state, which is encoded on tape 3, and the
  current symbol on Ts tape whose encoding starts
  in the current position on tape 2.
• In order to simulate this move, TU will search
  tape 1 for the 5-tuple whose first two parts
  match this state-input combination. The remaining
  parts of the 5 tuple tell us how to simulate the
  move -- specify the change in the tape symbol,
  the move of direction and the state change).

25
Church Turing Thesis (CT Thesis)

• Simply stated, the CT Thesis simply states that a
  TM is a perfect model of computation.
• What do we mean by that? We mean that any
  algorithmic procedure can be carried out by a
  turing machine. It is not a mathematically
  precise statement in that we dont define
  algorithmic procesure.
• However, there is plenty of evidence to support
  the CT Thesis
• The TM was constructed by considering what a
  human following some algorithm might do.
• Various enhancements to the TM (doubly-infinite
  tape, multiple tape heads, nondeterminism) do NOT
  change the computing power of the TM.
• Other theoretical models of computation that have
  been proposed have either been shown to be
  equivalent to TMs, or to be less or more
  powerful. (less powerful algorithmic
  procedures they cant handles, more powerful
  problems they can solve that algorithmic
  procedures cant)
• No problem has ever been described that is
  considered an algorithmic procedure that cannot
  be solved on a TM.

26

• The CT Thesis is not something we can prove
  because we dont have a precise statement of what
  algorithmic procedure means. However, once we
  adopt the CT thesis, we CAN precisely define an
  algorithmic procedure.
• Def An algorithm is a procedure that can be
  executed on a TM.
• For example, if a TM can compute f(x), there we
  say there is an algorithm for computing f(x)