Theory of Computation

1
Theory of Computation
2
Computation

• Computation is a general term for any type of
  information processing that can be represented as
  an algorithm precisely (mathematically).

3
Computation

• Computation is a general term for any type of
  information processing that can be represented as
  an algorithm precisely (mathematically).
• Examples
• Adding two numbers in our brains, on a piece of
  paper or using a calculator.

4
Computation

• Computation is a general term for any type of
  information processing that can be represented as
  an algorithm precisely. (mathematically)
• Examples
• Adding two numbers in our brains, on a piece of
  paper or using a calculator.
• Converting a decimal number to its binary
  presentation or vise versa.

5
Computation

• Computation is a general term for any type of
  information processing that can be represented as
  an algorithm precisely (mathematically).
• Examples
• Adding two numbers in our brains, on a piece of
  paper or using a calculator.
• Converting a decimal number to its binary
  presentation or vise versa.
• Finding the greatest common divisors of two
  numbers.

6
Theory of Computation

• A very fundamental and traditional branch of
  Theory of Computation seeks

7
Theory of Computation

• A very fundamental and traditional branch of
  Theory of Computation seeks
• A more tangible definition for the intuitive
  notion of algorithm which results in a more
  concrete definition for computation.

8
Theory of Computation

• A very fundamental and traditional branch of
  Theory of Computation seeks
• A more tangible definition for the intuitive
  notion of algorithm which results in a more
  concrete definition for computation.
• Finding the boundaries (limitations) of
  computation.

9
Algorithm

• A finite sequence of simple instructions that is
  guaranteed to halt in a finite amount of time.

10
Algorithm

• A finite sequence of simple instructions that is
  guaranteed to halt in a finite amount of time.
• This is a very abstract definition, since
• We didnt specify the nature of this simple
  instructions.
• For example an instruction can be increment a
  number by one or Calculate the triple integral

11
Algorithm

• A finite sequence of simple instructions that is
  guaranteed to halt in a finite amount of time.
• This is a very abstract definition, since
• We didnt specify the nature of this simple
  instructions.
• For example an instruction can be increment a
  number by one or Calculate the triple integral
• We didnt specify the entity which can execute
  these instructions.

12
Algorithm

• A finite sequence of simple instructions that is
  guaranteed to halt in a finite amount of time.
• This is a very abstract definition, since
• We didnt specify the nature of this simple
  instructions.
• For example an instruction can be increment a
  number by one or Calculate the triple integral
• We didnt specify the entity which can execute
  these instructions.
• For example is this entity a person, a computer,
  
• If it is a computer what is the processor type?
  How much memory does it have? . ?

13
An Abstract Machine

• To make a more solid definition of algorithm we
  need to define an abstract (general) machine
  which can perform any algorithm that can be
  executed by any computer.

14
An Abstract Machine

• To make a more solid definition of algorithm we
  need to define an abstract (general) machine
  which can perform any algorithm that can be
  executed by any computer.
• Then, We need to show that indeed this machine
  can run any algorithm that can be executed by any
  other computer. Then,

15
An Abstract Machine

• To make a more solid definition of algorithm we
  need to define an abstract (general) machine
  which can perform any algorithm that can be
  executed by any computer.
• Then, We need to show that indeed this machine
  can run any algorithm that can be executed by any
  other computer. Then,
• We can associate the notion of algorithm with
  this abstract machine.
• We can study this machine to find the limitations
  of computations. (Problems with no computation
  available to solve.)

16
Turing Machine

• A conceptual model for general purpose computers
  proposed by Alan Turing in 1936.

17
Turing Machine

• A conceptual model for general purpose computers
  proposed by Alan Turing in 1936.
• A Turing machine has an unlimited and
  unrestricted amount of memory.

18
Turing Machine

• A conceptual model for general purpose computers
  proposed by Alan Turing in 1936.
• A Turing machine has an unlimited and
  unrestricted amount of memory.
• A Turing machine can do everything a real
  computer can do.

19
Turing Machine

• A conceptual model for general purpose computers
  proposed by Alan Turing in 1936.
• A Turing machine has an unlimited and
  unrestricted amount of memory.
• A Turing machine can do everything a real
  computer can do.
• Nevertheless there are problems that a Turing
  machine cannot solve.

20
Turing Machine

• A conceptual model for general purpose computers
  proposed by Alan Turing in 1936.
• A Turing machine has an unlimited and
  unrestricted amount of memory.
• A Turing machine can do everything a real
  computer can do.
• Nevertheless there are problems that a Turing
  machine cannot solve.
• In a real sense, these problems are beyond the
  theoretical limits of computations.

21
Turing Machine Specification

• Components of Turing Machine
• An unlimited length tape of discrete cells.

22
Turing Machine Specification

• Components of Turing Machine
• An unlimited length tape of discrete cells.
• A head which reads and writes on tape.

23
Turing Machine Specification

• Components of Turing Machine
• An unlimited length tape of discrete cells.
• A head which reads and writes on tape.
• A control device with a finite number of states
  which can

24
Turing Machine Specification

• Components of Turing Machine
• An unlimited length tape of discrete cells.
• A head which reads and writes on tape.
• A control device with a finite number of states
  which can
• Instruct the head to read the symbol on the tape
  currently under head.

25
Turing Machine Specification

• Components of Turing Machine
• An unlimited length tape of discrete cells.
• A head which reads and writes on tape.
• A control device with a finite number of states
  which can
• Instruct the head to read the symbol on the tape
  currently under head.
• Instruct the head to write a symbol on the cell
  of the tape currently under tape.

26
Turing Machine Specification

• Components of Turing Machine
• An unlimited length tape of discrete cells.
• A head which reads and writes on tape.
• A control device with a finite number of states
  which can
• Instruct the head to read the symbol on the tape
  currently under head.
• Instruct the head to write a symbol on the cell
  of the tape currently under tape.
• Move the head one cell to left or right.

27
Turing Machine Specification

• Components of Turing Machine
• An unlimited length tape of discrete cells.
• A head which reads and writes on tape.
• A control device with a finite number of states
  which can
• Instruct the head to read the symbol on the tape
  currently under head.
• Instruct the head to write a symbol on the cell
  of the tape currently under tape.
• Move the head one cell to left or right.
• Change its current state.

28
A Turning Machine
29
Turing Machine Instructions

• Instructions of Turing Machine have the following
  format
• (Current State, Current Symbol, Write, Move
  L/R or No move, New State)
• Ex
• (2, 0, 1, L, 3)
• (3, 1, blank, N, 4)
• (1, , 0, R, 7)

30
Turing Machine Instructions

• The interpretation of the TM (Turing Machine)
  instructions
• (2, 0, 1, L, 3)
• When Turing machine (the control unit of
  TM) is at state 2 and the current tape symbol is
  0, write symbol 1 at current tape cell and go to
  state 3.

31
Visualization of TM instruction (2, 0, 1, L, 3)
32
Visualization of TM instruction (2, 0, 1, L, 3)
33
TM Conventions

• We always use state 1 as the initial state. (That
  is the execution of the algorithm or program
  begins with stating of the TM being 1.

34
TM Conventions

• We always use state 1 as the initial state. (That
  is the execution of the algorithm or program
  begins with stating of the TM being 1.
• The tape is used for recording input and output,
  one symbol per cell. Initially, the string to
  serve as input to our computation is recorded
  beginning from the leftmost tape cell.

35
TM Conventions

• We always use state 1 as the initial state. (That
  is the execution of the algorithm or program
  begins with stating of the TM being 1.
• The tape is used for recording input and output,
  one symbol per cell. Initially, the string to
  serve as input to our computation is recorded
  beginning from the leftmost tape cell.
• Initially, the position of head is at left most
  cell.

36
Initial Configuration of TM
37
The Output of TM

• The output of a TM program or algorithm is the
  sequence of symbols on the tape when the TM halts
  on that program.

38
TM Programs

• A Turing machine program is a set of TM
  instructions.

39
TM Programs

• A Turing machine program is a set of TM
  instructions.
• Turing machine halts on a program if there is no
  instruction in the program which its current
  state is the current state of the machine and its
  current symbol is the current symbol of the tape
  of the machine (symbol under head of the machine).

40
Example 1

• (1, 1, 1, R, 2), (2, 1, 1, R, 2),
• (2, blank, blank, R, 3), (3, 1, blank, L, 4),
• (4, blank, 1, R, 2)

41
Example 1

• (1, 1, 1, R, 2), (2, 1, 1, R, 2),
• (2, blank, blank, R, 3), (3, 1, blank, L, 4),
• (4, blank, 1, R, 2)
• This program outputs the sum of two integers m
  and n given as input.

42
Example 1

• (1, 1, 1, R, 2), (2, 1, 1, R, 2),
• (2, blank, blank, R, 3), (3, 1, blank, L, 4),
• (4, blank, 1, R, 2)
• This program outputs the sum of two integers m
  and n given as input.
• The numbers are in base 1 (unary notation).

43
Example 1

• (1, 1, 1, R, 2), (2, 1, 1, R, 2),
• (2, blank, blank, R, 3), (3, 1, blank, L, 4),
• (4, blank, 1, R, 2)
• This program outputs the sum of two integers m
  and n given as input.
• The numbers are in base 1 (unary notation).
• Examples of integers in unary notation
• 1 1 2 11 3 111 4 1111 .
• number n n number of 1s.

44
Example 1

• The input on tape (the initial configuration)
• 1 1 b 1 1 1 1 b b b state 1
• Inputs operands 2 and 4.

45
Example 1

• The input on tape (the initial configuration)
• 1 1 b 1 1 1 1 b b b state 1
• Inputs operands 2 and 4.
• The output on tape (when the program halts)
• 1 1 1 1 1 1 b b b state 3
• output 6
• b stands for blank.

46
Example 1

• Executing the program

47
Example 1

• 1 1 b 1 1 1 1 b b state 1
• Instruction which is going to be executed
• (1, 1, 1, R, 2)

48
Example 1

• 1 1 b 1 1 1 1 b b state 2
• Instruction which is going to be executed
• (2, 1, 1, R, 2)

49
Example 1

• 1 1 b 1 1 1 1 b b state 2
• Instruction which is going to be executed
• (2, blank, blank, R, 3)

50
Example 1

• 1 1 b 1 1 1 1 b b state 3
• Instruction which is going to be executed
• (3, 1, blank, L, 4)

51
Example 1

• 1 1 b b 1 1 1 b b state 4
• Instruction which is going to be executed
• (4, blank, 1, R, 2)

52
Example 1

• 1 1 1 b 1 1 1 b b state 2
• Instruction which is going to be executed
• (2, blank, blank, R, 3),

53
Example 1

• 1 1 1 b 1 1 1 b b state 3
• Instruction which is going to be executed
• (3, 1, blank, L, 4)

54
Example 1

• 1 1 1 b b 1 1 b b state 4
• Instruction which is going to be executed
• (4, blank, 1, R, 2)

55
Example 1

• 1 1 1 1 b 1 1 b b state 2
• Instruction which is going to be executed
• (2, blank, blank, R, 3)

56
Example 1

• 1 1 1 1 b 1 1 b b state 3
• Instruction which is going to be executed
• (3, 1, blank, L, 4)

57
Example 1

• 1 1 1 1 b b 1 b b state 4
• Instruction which is going to be executed
• (4, blank, 1, R, 2)

58
Example 1

• 1 1 1 1 1 b 1 b b state 2
• Instruction which is going to be executed
• (2, blank, blank, R, 3)

59
Example 1

• 1 1 1 1 1 b 1 b b state 3
• Instruction which is going to be executed
• (3, 1, blank, L, 4)

60
Example 1

• 1 1 1 1 1 b b b b state 4
• Instruction which is going to be executed
• (4, blank, 1, R, 2)

61
Example 1

• 1 1 1 1 1 1 b b b state 2
• Instruction which is going to be executed
• (2, blank, blank, R, 3)

62
Example 1

• 1 1 1 1 1 1 b b b state 3
• There is no instruction starting with
• (3 ,blank , . ) gt HALT
• Output 1 1 1 1 1 1 b b b

63
Example 1

• 1 1 1 1 1 1 b b b state 3
• There is no instruction starting with
• (3 ,blank , . ) gt HALT
• Output 1 1 1 1 1 1 b b b
• What is the function computed by this TM
  prorgram?

64
Example 2

• (1, 0, 0, R, 2), (1, 1, 1, R, 2), (2, 0, 0, R,
  2), (2, 1, 1, R, 2), (2, blank, 0, R, 3), (3,
  blank, 0, R, 4 )
• Number of states 4
• Used alphabet 0, 1

65
Example 2

• 1 1 0 0 1 0 1 1 b b b state 1
• (1, 0, 0, R,
  2)
• gt (1, 1, 1, R,
  2)
• (2, 0, 0, R,
  2)
• (2, 1, 1, R,
  2)
• (2, blank, 0,
  R, 3)
• (3, blank, 0,
  R, 4 )

66
Example 2

• 1 1 0 0 1 0 1 1 b b b state 2
• (1, 0, 0, R,
  2)
• (1, 1, 1, R,
  2)
• (2, 0, 0, R,
  2)
• gt (2, 1, 1, R,
  2)
• (2, blank, 0,
  R, 3)
• (3, blank, 0,
  R, 4 )

67
Example 2

• 1 1 0 0 1 0 1 1 b b b state 2
• (1, 0, 0, R,
  2)
• (1, 1, 1, R,
  2)
• gt (2, 0, 0, R,
  2)
• (2, 1, 1, R,
  2)
• (2, blank, 0,
  R, 3)
• (3, blank, 0,
  R, 4 )

68
Example 2

• 1 1 0 0 1 0 1 1 b b b state 2
• (1, 0, 0, R,
  2)
• (1, 1, 1, R,
  2)
• gt (2, 0, 0, R,
  2)
• (2, 1, 1, R,
  2)
• (2, blank, 0,
  R, 3)
• (3, blank, 0,
  R, 4 )

69
Example 2

• 1 1 0 0 1 0 1 1 b b b state 2
• (1, 0, 0, R,
  2)
• (1, 1, 1, R,
  2)
• (2, 0, 0, R,
  2)
• gt (2, 1, 1, R,
  2)
• (2, blank, 0,
  R, 3)
• (3, blank, 0,
  R, 4 )

70
Example 2

• 1 1 0 0 1 0 1 1 b b b state 2
• (1, 0, 0, R,
  2)
• (1, 1, 1, R,
  2)
• gt (2, 0, 0, R,
  2)
• (2, 1, 1, R,
  2)
• (2, blank, 0,
  R, 3)
• (3, blank, 0,
  R, 4 )

71
Example 2

• 1 1 0 0 1 0 1 1 b b b state 2
• (1, 0, 0, R,
  2)
• (1, 1, 1, R,
  2)
• (2, 0, 0, R,
  2)
• gt (2, 1, 1, R,
  2)
• (2, blank, 0,
  R, 3)
• (3, blank, 0,
  R, 4 )

72
Example 2

• 1 1 0 0 1 0 1 1 b b b state 2
• (1, 0, 0, R,
  2)
• (1, 1, 1, R,
  2)
• (2, 0, 0, R,
  2)
• gt (2, 1, 1, R,
  2)
• (2, blank, 0,
  R, 3)
• (3, blank, 0,
  R, 4 )

73
Example 2

• 1 1 0 0 1 0 1 1 b b b state 2
• (1, 0, 0, R,
  2)
• (1, 1, 1, R,
  2)
• (2, 0, 0, R,
  2)
• (2, 1, 1, R,
  2)
• gt (2, blank, 0,
  R, 3)
• (3, blank, 0,
  R, 4 )

74
Example 2

• 1 1 0 0 1 0 1 1 0 b b state 3
• (1, 0, 0, R,
  2)
• (1, 1, 1, R,
  2)
• (2, 0, 0, R,
  2)
• (2, 1, 1, R,
  2)
• (2, blank, 0,
  R, 3)
• gt (3, blank, 0,
  R, 4 )

75
Example 2

• 1 1 0 0 1 0 1 1 0 0 b state 4
• (1, 0, 0, R,
  2)
• (1, 1, 1, R,
  2)
• (2, 0, 0, R,
  2)
• (2, 1, 1, R,
  2)
• (2, blank, 0,
  R, 3)
• (3, blank, 0,
  R, 4 )

76
Example 2

• HALT. Output
• 1 1 0 0 1 0 1 1 0 0 b state 4
• INPUT
• 11001011
• What is the function that is being computed by
  this program?

77
Example 2

• OUTPUT 1100101100
• INPUT 11001011
• Input is base-2 presentation of number
• 203 and output is the base-2
• presentation of number 812.

78
Example 2

• OUTPUT 1100101100
• INPUT 11001011
• Input is base-2 presentation of number
• 203 and output is the base-2
• presentation of number 812.
• Thus,
• f(x) 4x

79
The definition of Algorithm

• We have reasons to believe (Although we will not
  provide the reasoning here in this course) that
  for any algorithm (finite sequence of steps which
  stops in a finite amount of time) that can be
  executed on any machine, there is a TM algorithm
  (program) which can be executed on TM and
  performs the same action.

80
Conclusion

• Intuitive notion of
  Turing machine
• algorithm equals
  algorithm
• The Church-Turing Thesis

81
Decidable Problems

• Problems, for which we cant find an algorithm
  that answer all possible instances of the
  problem.

82
Decidable Problems

• Problems, for which we cant find an algorithm
  that answer all possible instances of the
  problem.
• That is there is no TM program which answer all
  possible instances of the problem in a finite
  amount of time.

83
Decidable Problems

• For a decidable problem there is a program such
  that if an instance of the problem has solution,
  the program eventually halts with answer. But if
  there is no solution for that instance, the
  program will not ever halt.

84
Decidable Problems

• For a decidable problem there is a program such
  that if an instance of the problem has solution,
  the program eventually halts with answer. But if
  there is no solution for that instance, the
  program will not ever halt.
• Can we consider such programs as algorithms?

85
Decidable Problems

• For a decidable problem there is a program such
  that if an instance of the problem has solution,
  the program eventually halts with answer. But if
  there is no solution for that instance, the
  program will not ever halt.
• Can we consider such programs as algorithms?
• Answer No, because they might not halt.

86
An Un-decidable Problem

• The problem of finding an integral solution for a
  collection of multi-variable polynomial
  equations, is not decidable.
• For example consider the following two instances
  of problem

87
Examples
88
Examples

• Assume, we have a program which assigns all
  possible combination of 3 integers to variables
  x, y and z. For the first case there is at least
  one solution (x 2, y 1, z 5). Thus, the
  program will eventually stops. But for the second
  case we dont know if this system has a solution.
  If there is no solution for the second system,
  then the program never stops.