Module 12

1
Module 12

• Computation and Configurations
• Formal Definition
• Examples

2
Definitions

• Configuration
• Functional Definition
• Given the original program and the current
  configuration of a computation, someone should be
  able to complete the computation
• Contents of a configuration for a C program
• current instruction to be executed
• current value of all variables
• Computation
• Complete sequence of configurations

3
Computation 1

• 1 int main(int x,y)
• 2 int r x y
• 3 if (r 0) goto 8
• 4 x y
• 5 y r
• 6 r x y
• 7 goto 3
• 8 return y
• Input 10 3

• Line 1, x?,y?,r?
• Line 2, x10, y3,r?
• Line 3, x10, y3, r1
• Line 4, x10, y3, r1
• Line 5, x 3, y3, r1
• Line 6, x3, y1, r1
• Line 7, x3, y1, r0
• Line 3, x3, y1, r0
• Line 8, x3, y1, r0
• Output is 1

4
Computation 2

• int main(int x,y)
• 2 int r x y
• 3 if (r 0) goto 8
• 4 x y
• 5 y r
• 6 r x y
• 7 goto 3
• 8 return y
• Input 53 10

• Line 1, x?,y?,r?
• Line 2, x53, y10, r?
• Line 3, x 53, y10, r3
• Line 4, x53, y10, r3
• Line 5, x10, y10, r3
• Line 6, x10, y3, r3
• Line 7, x10, y3, r1
• Line 3, x10, y3, r1
• ...

5
Computations 1 and 2

• Line 1, x?,y?,r?
• Line 2, x53, y10, r?
• Line 3, x 53, y10, r3
• Line 4, x53, y10, r3
• Line 5, x10, y10, r3
• Line 6, x10, y3, r3
• Line 7, x10, y3, r1
• Line 3, x10, y3, r1
• ...

• Line 1, x?,y?,r?
• Line 2, x10, y3,r?
• Line 3, x10, y3, r1
• Line 4, x10, y3, r1
• Line 5, x 3, y3, r1
• Line 6, x3, y1, r1
• Line 7, x3, y1, r0
• Line 3, x3, y1, r0
• Line 8, x3, y1, r0
• Output is 1

6
Observation

• int main(int x,y)
• 2 int r x y
• 3 if (r 0) goto 8
• 4 x y
• 5 y r
• 6 r x y
• 7 goto 3
• 8 return y
• Line 3, x 10, y3, r1

• Program and current configuration
• Together, these two pieces of information are
  enough to complete the computation
• Are they enough to determine what the original
  input was?
• No!
• Both previous inputs, 10 3 as well as 53 10
  eventually reached the same configuration (Line
  3, x10, y3, r1)

7
Module 13

• Studying the internal structure of REC, the set
  of solvable problems
• Complexity theory overview
• Automata theory preview
• Motivating Problem
• string searching

8
Studying REC

• Complexity Theory
• Automata Theory

9
Current picture of all languages
All Languages
REC Solvable
RE-REC
All languages - RE
Half Solvable
Not even half solvable
Which language class should be studied further?
10
Complexity Theory
REC
RE - REC
All languages - RE

• In complexity theory, we differentiate problems
  by how hard a problem is to solve
• Remember, all problems in REC are solvable
• Which problem is harder and why?
• Max
• Input list of n numbers
• Task return largest of the n numbers
• Element
• Input list of n numbers
• Task return any of the n numbers

11
Resource Usage

• How do we formally measure the hardness of a
  problem?
• We measure the resources required to solve input
  instances of the problem
• Typical resources are?
• Need a notion of size of an input instance
• Obviously larger input instances require more
  resources to solve

12
Poly Language Class
REC
RE - REC
All languages - RE
Poly
Rest of REC
Informal Definition A problem L1 is easier than
problem L2 if problem L1 can be solved in less
time than problem L2. Poly the set of problems
which can be solved in polynomial time (typically
referred to as P, not Poly) Major goal Identify
whether or not a problem belongs to Poly
13
Working with Poly
Poly
Rest of REC

• How do you prove a problem L is in Poly?
• How do you prove a problem L is not in Poly?
• We are not very good at this.
• For a large class of interesting problems, we
  have techniques (polynomial-time
  answer-preserving input transformations) that
  show a problem L probably is not in Poly, but
  few which prove it.

14
Examples
Poly
Rest of REC

• Shortest Path Problem
• Input
• Graph G
• nodes s and t
• Task
• Find length of shortest path from s to t in G

• Longest Path Problem
• Input
• Graph G
• nodes s and t
• Task
• Find length of longest path from s to t in G

Which problem is provably solvable in polynomial
time?
15
Automata Theory
REC
RE - REC
All languages - RE

• In automata theory, we will define new models of
  computation which we call automata
• Finite State Automata (FSA)
• Pushdown Automata (PDA)
• Key concept
• FSAs and PDAs are restricted models of
  computation
• FSAs and PDAs cannot solve all the problems
  that C programs can
• We then identify which problems can be solved
  using FSAs and PDAs

16
New language classes
REC
RE - REC
All languages - RE

• REC is the set of solvable languages when we
  start with a general model of computation like
  C programs
• We want to identify which problems in REC can be
  solved when using these restricted automata

Rest of REC
17
Recap

• Complexity Theory
• Studies structure of the set of solvable problems
• Method analyze resources (processing time) used
  to solve a problem
• Automata Theory
• Studies structure of the set of solvable problems
• Method define automata with restricted
  capabilities and resources and see what they can
  solve (and what they cannot solve)
• This theory also has important implications in
  the development of programming languages and
  compilers

18
Motivating Problem

• String Searching

19
String Searching

• Input
• String x
• String y
• Tasks
• Return location of y in string x
• Does string y occur in string x?

• Can you identify applications of this type of
  problem in real life?
• Try and develop a solution to this problem.

20
String Searching II

• Input
• String x
• pattern y
• Tasks
• Return location of y in string x
• Does pattern y occur in string x?

• Pattern
• anything.html
• EN4
• Try and develop a solution to this problem.

21
String Searching

• We will show an easy way to solve these string
  searching problems
• In particular, we will show that we can solve
  these problems in the following manner
• write down the pattern
• the computer automatically turns this into a
  program which performs the actual string search

22
Module 14

• Regular languages
• Inductive definitions
• Regular expressions
• syntax
• semantics

23
Regular Languages(Regular Expressions)
24
Regular Languages

• New language class
• Elements are languages
• We will show that this language class is
  identical to LFSA
• Language class to be defined by Finite State
  Automata (FSA)
• Once we have shown this, we will use the term
  regular languages to refer to this language
  class

25
Inductive Definition of Integers

• Base case definition
• 0 is an integer
• Inductive case definition
• If x is an integer, then
• x1 is an integer
• x-1 is an integer
• Completeness
• Only numbers generated using the above rules are
  integers

26
Inductive Definition of Regular Languages

• Base case definition
• Let S denote the alphabet
• is a regular language
• l is a regular language
• a is a regular language for any character a in
  S
• Inductive case definition
• If L1 and L2 are regular languages, then
• L1 union L2 is a regular language
• L1 concatenate L2 is a regular language
• L1 is a regular language
• Completeness
• Only languages generated using above rules are
  regular languages

27
Proving a language is regular

• Prove that aa, bb is a regular language
• a and b are regular languages
• base case of definition
• aa aa is a regular language
• concatenation rule
• bb bb is a regular language
• concatenation rule
• aa, bb aa union bb is a regular language
• union rule
• Typically, we will not go through this process to
  prove a language is regular

28
Regular Expressions

• How do we describe a regular language?
• Use set notation
• aa, bb, ab, ba
• aa,bb
• Use regular expressions R
• Inductive def of regular languages and regular
  expressions on page 72
• (aabbabba)
• a(ab)b

29
R and L(R)

• How we interpret a regular expression
• What does a regular expression R mean to us?
• aaba represents the regular language aaba
• f represents the regular language
• aabb represents the regular language aa, bb
• We use L(R) to denote the regular language
  represented by regular expression R.

30
Precedence rules

• What is L(abc)?
• Possible answers
• a(b union c
• (ab,c)
• (ab union c)
• ab union c
• Must know precedence rules
• first, then concatenation, then

31
Precedence rules continued

• Precedence rules similar to those for arithmetic
  expressions
• abc2
• (a times b) (c times c)
• exponentiation first, then multiplication, then
  addition
• Think of Kleene closure as exponentiation,
  concatenation as multiplication, and union as
  addition and the precedence rules are identical

32
Regular expressions are strings

• Let L be a regular language over the alphabet S
• A regular expression R for L is just a string
  over the alphabet S union (, ), , , f, l
  which follows certain syntactic rules
• That is, the set of legal regular expressions is
  itself a language over the alphabet S union (,
  ), ,
• f, aaba are strings in the language of legal
  reg. exp.
• )(, a are strings NOT in the language of legal
  reg. exp.

33
Semantics

• We give a regular expression R meaning when we
  interpret it to represent L(R).
• aaba is just a string
• we interpret it to represent the language
  aaba.
• We do similar things with arithmetic expressions
• 1072 is just a string
• We interpret this string to represent the number
  59

34
Key fact

• A language L is a regular language iff there
  exists a reg. exp. R such that L(R) L
• When I ask for a proof that a language L is
  regular, rather than going through the inductive
  proof we saw earlier, I expect you to give me a
  regular expression R s.t. L(R) L

35
Summary

• Regular expressions are strings
• syntax for legal regular expressions
• semantics for interpreting regular expressions
• Regular languages are a new language class
• A language L is regular iff there exists a
  regular expression R s.t. L(R) L
• We will show that the regular languages are
  identical to LFSA

36
Module 15

• FSAs
• Defining FSAs
• Computing with FSAs
• Defining L(M)
• Defining language class LFSA
• Comparing LFSA to set of solvable languages (REC)

37
Finite State Automata

• New Computational Model

38
Tape

• We assume that you have already seen FSAs in CSE
  260
• If not, review material in reference textbook
• Only data structure is a tape
• Input appears on tape followed by a B character
  marking the end of the input
• Tape is scanned by a tape head that starts at
  leftmost cell and always scans to the right

39
Data type/States

• The only data type for an FSA is char
• The instructions in an FSA are referred to as
  states
• Each instruction can be thought of as a switch
  statement with several cases based on the char
  being scanned by the tape head

40
Example program

• 1 switch(current tape cell)
• case a goto 2
• case b goto 2
• case B return yes
• 2 switch (current tape cell)
• case a goto 1
• case b goto 1
• case B return no

41
New model of computation

• FSA M(Q,S,q0,A,d)
• Q set of states 1,2
• S character set a,b
• dont need B as we see below
• q0 initial state 1
• A set of accepting (final) states 1
• A is the set of states where we return yes on B
• Q-A is set of states that return no on B
• d state transition function

• 1 switch(current tape cell)
• case a goto 2
• case b goto 2
• case B return yes
• 2 switch (current tape cell)
• case a goto 1
• case b goto 1
• case B return no

42
Textual representations of d
d(1,a) 2, d(1,b)2, d(2,a)1, d(2,b) 1

• 1 switch(current tape cell)
• case a goto 2
• case b goto 2
• case B return yes
• 2 switch (current tape cell)
• case a goto 1
• case b goto 1
• case B return no

(1,a,2), (1,b,2), (2,a,1), (2,b,1)
43
Transition diagrams

• 1 switch(current tape cell)
• case a goto 2
• case b goto 2
• case B return yes
• 2 switch (current tape cell)
• case a goto 1
• case b goto 1
• case B return no

Note, this transition diagram represents all 5
components of an FSA, not just the transition
function d
44
Exercise

• FSA M (Q, S, q0, A, d)
• Q 1, 2, 3
• S a, b
• q0 1
• A 2,3
• d d(1,a) 1, d(1,b) 2, d(2,a) 2, d(2,b)
  3, d(3,a) 3, d(3,b) 1
• Draw this FSA as a transition diagram

45
Transition Diagram
46
Computing with FSAs
47
Computation Example
Input aabbaa
48
Computation of FSAs in detail

• A computation of an FSA M on an input x is a
  complete sequence of configurations
• We need to define
• Initial configuration of the computation
• How to determine the next configuration given the
  current configuration
• Halting or final configurations of the computation

49
Initial Configuration

• Given an FSA M and an input string x, what is the
  initial configuration of the computation of M on
  x?
• (q0,x)
• Examples
• x aabbaa
• (1, aabbaa)
• x abab
• (1, abab)
• x l
• (1, l)

FSA M
50
Definition of M

• (1, aabbaa) M (1, abbaa)
• config 1 yields config 2 in one step using FSA
  M
• (1,aabbaa) M (2, baa)
• config 1 yields config 2 in 3 steps using FSA M
• (1, aabbaa) M (2, baa)
• config 1 yields config 2 in 0 or more steps
  using FSA M
• Comment
• M determined by transition function d
• There must always be one and only one next
  configuration
• If not, M is not an FSA

3

FSA M
51
Halting Configurations

• Halting configuration
• (q, l)
• Examples
• (1, l)
• (3, l)
• Accepting Configuration
• State in halting configuration is in A
• Rejecting Configuration
• State in halting configuration is not in A

FSA M
52
FSA M on x

• Two possibilities for M running on x
• M accepts x
• M accepts x iff the computation of M on x ends up
  in an accepting configuration
• (q0, x) M (q, l) where q is in A
• M rejects x
• M rejects x iff the computation of M on x ends up
  in a rejecting configuration
• (q0, x) M (q, l) where q is not in A
• M does not loop or crash on x
• Why?

53
Examples

• For the following input strings, does M accept or
  reject?
• l
• aa
• aabba
• aab
• babbb

54
Definition of d(q, x)

• Notation from the book
• d(q, c) p
• dk(q, x) p
• k is the length of x
• d(q, x) p
• Examples
• d(1, a) 1
• d(1, b) 2
• d4(1, abbb) 1
• d(1, abbb) 1
• d(2, baaaaa) 3

FSA M
55
L(M) and LFSA

• L(M) or Y(M)
• The set of strings M accepts
• Basically the same as Y(P) from previous unit
• We say that M accepts/decides/recognizes/solves
  L(M)
• Remember an FSA will not loop or crash
• What is L(M) (or Y(M)) for the FSA M above?
• N(M)
• Rarely used, but it is the set of strings M
  rejects
• LFSA
• L is in LFSA iff there exists an FSA M such that
  L(M) L.

56
LFSA Unit Overview

• Study limits of LFSA
• Understand what languages are in LFSA
• Develop techniques for showing L is in LFSA
• Understand what languages are not in LFSA
• Develop techniques for showing L is not in LFSA
• Prove Closure Properties of LFSA
• Identify relationship of LFSA to other language
  classes

57
Comparing language classes

• Showing LFSA is a subset of REC, the set of
  solvable languages

58
LFSA subset REC

• Proof
• Let L be an arbitrary language in LFSA
• Let M be an FSA such that L(M) L
• M exists by definition of L in LFSA
• Construct C program P from FSA M
• Argue P solves L
• There exists a C program P which solves L
• L is solvable

59
Visualization

• Let L be an arbitrary language in LFSA
• Let M be an FSA such that L(M) L
• M exists by definition of L in LFSA
• Construct C program P from FSA M
• Argue P solves L
• There exists a program P which solves L
• L is solvable

LFSA
REC
60
Construction
FSA M
Program P
Construction Algorithm

• The construction is an algorithm which solves a
  problem with a program as input
• Input to A FSA M
• Output of A C program P such that P solves
  L(M)
• How do we do this?

61
Comparing computational models

• The previous slides show one method for comparing
  the relative power of two different computational
  models
• Computational model CM1 is at least as general or
  powerful as computational model CM2 if
• Any program P2 from computational model CM2 can
  be converted into an equivalent program P1 in
  computational model CM1.
• Question How can we show two computational
  models are equivalent?

62
Module 16

• Distinguishability
• Definition
• Help in designing/debugging FSAs

63
Distinguishability
64
Questions

• Let L be the set of strings over a,b which end
  with aaba.
• Let M be an FSA such that L(M) L.
• Questions
• Can aaba and aab end up in the same state of M?
  Why or why not?
• How about aa and aab?
• How about l or a?
• How about b or bb?
• How about l or bbab?

65
Definition

• String x is distinguishable from string y with
  respect to language L iff there exists a string z
  such that
• xz is in L and yz is not in L OR
• xz is not in L and yz is in L
• When reviewing, identify the z for pair of
  strings on the previous slide

66
Questions

• Let L be the set of strings over a,b that have
  length 2 mod 5 or 4 mod 5.
• Let M be an FSA such that L(M) L.
• Questions
• Are aa and aab distinguishable with respect to L?
  Can they end up in the same state of M?
• How about aa and aaba?
• How about l and a?
• How about b and aabbaa?

67
Design an FSA to accept L

• One design method
• Is l in L?
• Implication?
• Is a distinguishable from l wrt L?
• Implication?
• Is b distinguishable from l wrt L?
• Implication?
• Is b distinguishable from a wrt L?
• Implication?

• L set of strings x over a,b such that length
  of x is 2 or 4 mod 5

68
Design an FSA to accept L

• Design continued
• Is aa distinguishable from l wrt L?
• Implication?
• Is aa distinguishable from a wrt L?
• Implication?

• L set of strings x over a,b such that length
  of x is 2 or 4 mod 5

69
Design an FSA to accept L

• Design continued
• What strings would we compare ab to?
• What results do we get?
• Implications?
• How about ba?
• How about bb?

• L set of strings x over a,b such that length
  of x is 2 or 4 mod 5

70
Design an FSA to accept L

• Design continued
• We can continue in this vein, but it could go on
  forever
• Now lets try something different
• Consider string l.
• What set of strings are indistinguishable from it
  wrt L?
• Implications?

• L set of strings x over a,b such that length
  of x is 2 or 4 mod 5

71
Design an FSA to accept L

• Design continued
• Consider string a.
• What set of strings are indistinguishable from it
  wrt L?
• Implications?
• Consider string aa.
• What set of strings are indistinguishable from it
  wrt L?
• Implications?

• L set of strings x over a,b such that length
  of x is 2 or 4 mod 5

72
Debugging an FSA

• Do essentially the same thing
• Identify some strings which end up in each state
• Try and generalize each state to describe the
  language of strings which end up at that state.

73
Example 1
a,b
a
a
a
b
I
II
III
IV
V
b
a
b
b
VI
a,b
74
Example 2
a,b
b
a
I
II
III
IV
V
a
a
a
b
b
b
75
Example 3
a,b
II
a
I
IV
a
b
a
III
b
b
76
Module 17

• Closure Properties of Language class LFSA
• Remember ideas used in solvable languages unit
• Set complement
• Set intersection, union, difference, symmetric
  difference

77
LFSA is closed under set complement

• If L is in LFSA, then Lc is in LFSA
• Proof
• Let L be an arbitrary language in LFSA
• Let M be the FSA such that L(M) L
• M exists by definition of L in LFSA
• Construct FSA M from M
• Argue L(M) Lc
• There exists an FSA M such that L(M) Lc
• Lc is in LFSA

78
Visualization

• Let L be an arbitrary language in LFSA
• Let M be the FSA such that L(M) L
• M exists by definition of L in LFSA
• Construct FSA M from M
• Argue L(M) Lc
• Lc is in LFSA

LFSA
79
Construct FSA M from M

• What did we do when we proved that REC, the set
  of solvable languages, is closed under set
  complement?
• Construct program P from program P
• Can we translate this to the FSA setting?

80
Construct FSA M from M

• M (Q, S, q0, A, d)
• M (Q, S, q, A, d)
• M should say yes when M says no
• M should say no when M says yes
• How?
• Q Q
• S S
• q q0
• d d
• A Q-A

81
Example
Q Q S S q q0 d d A Q-A
a
a
2
b
1
b
b
3
FSA M
a
82
Construction is an algorithm
FSA M
FSA M
Construction Algorithm

• Set Complement Construction
• Algorithm Specification
• Input FSA M
• Output FSA M such that L(M) L(M)c
• Comments
• This algorithm can be in any computational model.
• It does not have to be (and typically is not) an
  FSA
• These set closure constructions are useful.
• More on this later

83
Specification of the algorithm
FSA M
FSA M
Construction Algorithm

• Your algorithm must give a complete specification
  of M in terms of M
• Example
• Let input FSA M (Q, S, q0, A, d)
• Output FSA M (Q, S, q, A, d) where
• Q Q
• S S
• q q0
• d d
• A Q-A
• When I ask for such a construction algorithm
  specification, this type of answer is what I am
  looking for. Further algorithmic details on how
  such an algorithm would work are unnecessary.

84
LFSA closed under Set Intersection Operation
(also set union, set difference, and symmetric
difference)
85
LFSA closed under set intersection operation

• Let L1 and L2 be arbitrary languages in LFSA
• Let M1 and M2 be FSAs s.t. L(M1) L1, L(M2)
  L2
• M1 and M2 exist by definition of L1 and L2 in
  LFSA
• Construct FSA M3 from FSAs M1 and M2
• Argue L(M3) L1 intersect L2
• There exists FSA M3 s.t. L(M3) L1 intersect L2
• L1 intersect L2 is in LFSA

86
Visualization

• Let L1 and L2 be arbitrary languages in LFSA
• Let M1 and M2 be FSAs s.t. L(M1) L1, L(M2)
  L2
• M1 and M2 exist by definition of L1 and L2 in
  LFSA
• Construct FSA M3 from FSAs M1 and M2
• Argue L(M3) L1 intersect L2
• There exists FSA M3 s.t. L(M3) L1 intersect L2
• L1 intersect L2 is in LFSA

LFSA
87
Algorithm Specification

• Input
• Two FSAs M1 and M2
• Output
• FSA M3 such that L(M3) L(M1) intersection L(M2)
  

FSA M1 FSA M2
FSA M3
88
Use Old Ideas

• Key concept Try ideas from previous closure
  property proofs
• Example
• How did the algorithm that was used to prove that
  REC is closed under set intersection work?
• If we adapt this approach, what should M3 do with
  respect to M1, M2, and the input string?

89
Run M1 and M2 Simultaneously
0
1
0
1
1
l
0
2
0
0
1
1
M1
What happens when M1 and M2 run on input string
11010?
90
Construction

• Input
• FSA M1 (Q1, S1, q1, d1, A1)
• FSA M2 (Q2, S2, q2, d2, A2)
• Output
• FSA M3 (Q3, S3, q3, d3, A3)
• What is Q3?
• Q3 Q1 X Q2 where X is cartesian product
• In this case, Q3 (l,A), (l,B), (0,A), (0,B),
  (1,A), (1,B), (2,A), (2,B)
• What is S3?
• S3 S1 S2
• In this case, S3 0,1

91
Construction

• Input
• FSA M1 (Q1, S1, q1, d1, A1)
• FSA M2 (Q2, S2, q2, d2, A2)
• Output
• FSA M3 (Q3, S3, q3, d3, A3)
• What is q3?
• q3 (q1, q2)
• In this case, q3 (l,A)
• What is A3?
• A3 (p, q) p in A1 and q in A2
• In this case, A3 (0,B)

92
Construction

• Input
• FSA M1 (Q1, S1, q1, d1, A1)
• FSA M2 (Q2, S2, q2, d2, A2)
• Output
• FSA M3 (Q3, S3, q3, d3, A3)
• What is d3?
• For all p in Q1, q in Q2, a in S, d3((p,q),a)
  (d1(p,a),d2(q,a))
• In this case,
• d3((0,A),0) (d1(0,0),d2(A,0))
• (0,B)
• d3((0,A),1) (d1(0,1),d2(A,1))
• (1,A)

93
Example Summary
1
0
1
1
0
1
1
l
0
2
0
l,A
0,A
1,A
2,A
1
0
1
1
0
0
0
M1
1
0
0
1
l,B
0,B
1,B
2,B
0
1
M3
94
Observation

• Input
• FSA M1 (Q1, S1, q1, d1, A1)
• FSA M2 (Q2, S2, q2, d2, A2)
• Output
• FSA M3 (Q3, S3, q3, d3, A3)
• What is A3?
• A3 (p, q) p in A1 and q in A2
• What if operation were different?
• Set union, set difference, symmetric difference

95
Observation continued

• Input
• FSA M1 (Q1, S1, q1, d1, A1)
• FSA M2 (Q2, S2, q2, d2, A2)
• Output
• FSA M3 (Q3, S3, q3, d3, A3)
• What is A3?
• Set intersection A3 (p, q) p in A1 and q in
  A2
• Set union A3 (p, q) p in A1 or q in A2
• Set difference A3 (p, q) p in A1 and q not
  in A2
• Symmetric difference A3 (p, q) (p in A1 and
  q not in A2) or (p not in A1 and q in A2)

96
Observation conclusion

• LFSA is closed under
• set intersection
• set union
• set difference
• symmetric difference
• The constructions used to prove these closure
  properties are essentially identical

97
Comments

• You should be able to execute this algorithm
• Convert two FSAs into a third FSA with the
  correct properties.
• You should understand the idea behind this
  algorithm
• The third FSA essentially runs both input FSAs
  simultaneously on any input string
• How we set A3 depending on the specific set
  operation
• You should understand how this algorithm can be
  used to simplify design of FSAs
• You should be able to construct new algorithms
  for new closure property proofs

98
Comparison
99
Module 18

• NFAs
• nondeterministic transition functions
• computations are trees, not paths
• L(M) and LNFA
• LFSA subset of LNFA
• Comparing computational models

100
Nondeterministic Finite State Automata

• NFAs

101
Change d is a relation

• For an FSA M, d(q,a) results in one and only one
  state for all states q and characters a.
• That is, d is a function
• For an NFA M, d(q,a) can result in a set of
  states
• That is, d is now a relation
• Next step is not determined (nondeterministic)

102
Example NFA

• Why is this only an NFA and not an FSA? Identify
  as many reasons as you can.

103
Computing with NFAs

• Configurations same as they are for FSAs
• Computations are different
• Initial configuration is identical
• However, there may be several next configurations
  or there may be none.
• Computation is no longer a path but is now a
  graph (often a tree) rooted at the initial
  configuration
• Definition of halting, accepting, and rejecting
  configurations is identical
• Definition of acceptance must be modified

104
Computation Graph (Tree)
Input string aaaaba
(1, aaaaba)
105
Definition of unchanged
Input string aaaaba
(1, aaaaba) (1, aaaba) (1, aaaaba) (2,
aaaba) (1, aaaaba)3 (1, aba) (1, aaaaba)3 (3,
aba) (1, aaaaba) (2, aba) (1, aaaaba) (3,
aba) (1, aaaaba) (1, l) (1, aaaaba) (5, l)
106
Acceptance and Rejection
Input string aaaaba
M accepts string x if one of the configurations
reached is an accepting configuration (q0, x)
(f, l),f in A M rejects string x if all
configurations reached are either not halting
configurations or are rejecting configurations
107
Comparison
a,b
b
a
a
a
a
b
b
b
FSA
108
Defining L(M) and LNFA

• M accepts string x if one of the configurations
  reached is an accepting configuration
• (q0, x) - (f, l),f in A
• M rejects string x if all configurations reached
  are either not halting configurations or are
  rejecting configurations

• L(M) (or Y(M))
• N(M)
• LNFA
• Language L is in language class LNFA iff

109
Comparing language classes

• LFSA subset of LNFA

110
LFSA subset LNFA

• Let L be an arbitrary language in LFSA
• Let M be the FSA such that L(M) L
• M exists by definition of L in LFSA
• Construct an NFA M such that L(M) L
• Argue L(M) L
• There exists an NFA M such that L(M) L
• L is in LNFA
• By definition of L in LNFA

111
Visualization

• Let L be an arbitrary language in LFSA
• Let M be an FSA such that L(M) L
• M exists by definition of L in LFSA
• Construct NFA M from FSA M
• Argue L(M) L
• There exists an NFA M such that L(M) L
• L is in LNFA

LFSA
LNFA
112
Construction

• We need to make M into an NFA M such that L(M)
  L(M)
• How do we accomplish this?

113
Module 19

• LNFA subset of LFSA
• Theorem 4.1 on page 131 of Martin textbook
• Compare with set closure proofs
• Main idea
• A state in FSA represents a set of states in
  original NFA

114
LNFA subset LFSA

• Let L be an arbitrary language in
• Let M be
• M exists by definition of
• Construct an M such that L(M)
• Argue L(M)
• There exists an M such that L(M)
• L is in
• By definition of

115
Visualization

• Let L be an arbitrary language in LNFA
• Let M be an NFA such that L(M) L
• M exists by definition of L in LNFA
• Construct FSA M from NFA M
• Argue L(M) L
• There exists an FSA M such that L(M) L
• L is in LFSA

LNFA
LFSA
116
Construction Specification

• We need an algorithm which does the following
• Input NFA M
• Output FSA M such that L(M) L(M)

117
Difficulty
Input string aaaaba

• An NFA can be in several states after processing
  an input string x

118
Observation
Input string aaaaba

• All strings which end up in the set of states
  1,2,3 are indistinguishable with respect to L(M)

119
Idea
Input string aaaaba

• Given an NFA M (Q,S,q0,d,A), the equivalent FSA
  M will have state set 2Q (one state for each
  subset of Q)
• Example
• In this case there are 5 states in Q
• 2Q, the set of all subsets of Q, has 25 elements
  including and Q
• The FSA M will have 25 states
• What strings end up in state 1,2,3 of M?
• The strings which end up in states 1, 2, and 3 of
  NFA M.
• In this case, strings which do not contain aaba
  and end with aa such as aa, aaa, and aaaa.

120
Idea Illustrated
Input string aaaaba
121
Construction
1
2
3

• Input NFA M (Q, S, q0, d, A)
• Output FSA M (Q, S, q, d, A)
• What is Q?
• all subsets of Q including Q and
• In this case, Q
• What is S?
• We always make S S
• In this case, S S a,b
• What is q?
• We always make q q0
• In this case q

122
Construction
1
2
3

• Input NFA M (Q, S, q0, d, A)
• Output FSA M (Q, S, q, d, A)
• What is A?
• Suppose a string x ends up in states 1 and 2 of
  the NFA M above.
• Is x accepted by M?
• Should 1,2 be an accepting state in FSA M?
• Suppose a string x ends up in states 1 and 2 and
  3 of the NFA M above.
• Is x accepted by M?
• Should 1,2,3 be an accepting state in FSA M?
• Suppose p q1, q2, , qk where q1, q2, , qk
  are in Q
• p is in A iff at least one of the states q1, q2,
  , qk is in A
• In this case, A

123
Construction
1
2
3

• Input NFA M (Q, S, q0, d, A)
• Output FSA M (Q, S, q, d, A)
• What is d?
• If string x ends up in states 1 and 2 after being
  processed by the NFA above, where does string xa
  end up after being processed by the NFA above?
• Figuring out d(p,a) in general
• Suppose p q1, q2, , qk where q1, q2, , qk
  are in Q
• Then d(p,a) d(q1,a) union d(q2,a) union
  union d(qk,a)
• Similar to 2 FSA to 1 FSA construction
• In this case
• d(1,2,a)

124
Construction Summary
1
2
3

• Input NFA M (Q, S, q0, d, A)
• Output FSA M (Q, S, q, d, A)
• Q all subsets of Q including Q and
• In this case, Q , 1, 2, 3, 1,2,
  1,3, 2,3, 1,2,3
• S S
• In this case, S S a,b
• q q0
• In this case, q 1
• A
• Suppose p q1, q2, , qk where q1, q2, , qk
  are in Q
• p is in A iff at least one of the states q1, q2,
  , qk is in A
• d
• Suppose p q1, q2, , qk where q1, q2, , qk
  are in Q
• Then d(p,a) d(q1,a) union d(q2,a) union
  union d(qk,a)

125
Example Summary
a,b
a,b
1
a
a
2
3
NFA M
126
Example Summary Continued
a
b
b
a,b
a,b
a
b
a
1
1,2
1,2,3
1,3
1
a
a
2
3
b
a
a,b
a,b
NFA M
a,b
a
b

2
3
2,3
FSA M
127
Example Summary Continued
a
b
b
a,b
a,b
a
b
a
1
1,2
1,2,3
1,3
1
a
a
2
3
b
a
NFA M
Smaller FSA M
By examination, we can see that state 1,3 is
unnecessary. However, this is a case by case
optimization. It is not a general technique or
algorithm.
128
Example 2
a,b
a
b
NFA M
Step 1 name the three states of NFA M
129
Step 2 transition table
a
b
A
B

d(B,C,a) d(B,a) U d(C,a)
B U B d(B,C,b)
d(B,b) U d(C,b) B,C U
B,C
B
B,C


B,C
B
B,C
130
Step 3 accepting states
a
b
Which states should be accepting? Any state which
includes an accepting state of M, in this case,
C. A B,C
A
B

B
B
B,C



B,C
B
B,C
131
Step 4 Answer
Initial state is A Set of final states A
B,C
This is sufficient. You do NOT need to turn this
into a diagram.
132
Step 5 Optional
FSA M
133
Comments

• You should be able to execute this algorithm
• You should be able to convert any NFA into an
  equivalent FSA.
• You should understand the idea behind this
  algorithm
• For an FSA M, strings which end up in the same
  state of M are indistinguishable wrt L(M)
• For an NFA M, strings which end up in the same
  set of states of M are indistinguishable wrt L(M)

134
Comments

• You should understand the importance of this
  algorithm
• Design tool
• We can design using NFAs
• A computer will convert this NFA into an
  equivalent FSA
• FSAs can be executed by computers whereas NFAs
  cannot (or at least cannot easily be run by
  computers)
• Chaining together algorithms
• Perhaps it is easy to build NFAs to accept L1
  and L2
• Use this algorithm to turn these NFAs to FSAs
• Use previous algorithm to build FSA to accept L1
  intersect L2
• You should be able to construct new algorithms
  for new closure property proofs

135
Module 20

• NFAs with l-transitions
• NFA-ls
• Formal definition
• Simplifies construction
• LNFA-l
• Showing LNFA-l is a subset of LNFA (extra credit)
• and therefore a subset of LFSA

136
Defining NFA-ls
137
Change l-transitions

• We now allow an NFA M to change state without
  reading input
• That is, we add the following categories of
  transitions to d
• d(q,l) is allowed

138
Example
139
Defining L(M) and LNFA-l

• M accepts string x if one of the configurations
  reached is an accepting configuration
• (q0, x) - (f, l),f e A
• M rejects string x if all configurations reached
  are either not halting configurations or are
  rejecting configurations

• L(M) or Y(M)
• N(M)
• LNFA-l
• Language L is in language class LNFA-l iff

140
LNFA-l subset LFSA

• Recap of what we already know
• Let M be any NFA
• There exists an algorithm A1 which constructs an
  FSA M such that L(M) L(M)
• New goal
• Let M be any NFA-l
• There exists an algorithm A2 which constructs an
  FSA M such that L(M) L(M)

141
Visualization

• Goal
• Let M be any NFA-l
• There exists an algorithm A2 which constructs an
  FSA M such that L(M) L(M)

NFA-l M
FSA M
142
Modified Goal

• Question
• Can we use any existing algorithms to simplify
  the task of developing algorithm A2?
• Yes, we can use algorithm A1 which converts an
  NFA M1 into an FSA M such that L(M) L(M1)

143
New Goal (extra credit)

• Difficulty
• NFA-l M can make transitions on l
• How can the NFA M1 simulate these l-transitions?

a
l
l
b
b
l
2
3
4
5
6
1
144
Basic Idea

• For each state q of M and each character a of S,
  figure out which states are reachable from q
  taking any number of l-transitions and exactly
  one transition on that character a.
• In the NFA-d M1, directly connect q to each of
  these states using an arc labeled with a.

a
l
l
b
b
l
2
3
4
5
6
1
2
3
4
5
6
1
145
Process State 2
a
l
l
b
b
l
2
3
4
5
6
1
2
3
4
5
6
1
146
Process State 3
a
l
l
b
b
l
2
3
4
5
6
1
2
3
4
5
6
1
147
Final Picture
a
a
a
b
a
2
3
4
5
6
1
b
b
148
Construction

• Input NFA-l M (Q, S, q0, d, A)
• Output NFA M1 (Q1, S1, q1, d1, A1)
• What is Q1?
• Q1 Q
• In this case, Q1 1,2,3,4,5,6
• What is S1?
• S1 S
• In this case, S1 S a,b
• What is q1?
• We always make q1 q0
• In this case q1 1

149
Construction

• Input NFA-l M (Q, S, q0, d, A)
• Output NFA M1 (Q1, S1, q1, d1, A1)
• What is d1?
• d1(q,a) the set of states reachable from state
  q in M taking any number of l-transitions and
  exactly one transition on the character a
• More on this later
• In this case
• d1(1,a)
• d1(1,b) 3,4,5
• What is A1?
• A1 A with one minor change
• If an accepting state is reachable from q0 using
  only l-transitions, then we make q1 an element of
  A1
• In this case, using only l-transitions, no
  accepting state is reachable from q0, so A1 A

150
Computing d1(q,a)

• d1(q,a) the set of states reachable from state
  q in M taking 0 or more l-transitions and exactly
  one transition on the character a
• Break this down into three steps
• First compute all states reachable from q using 0
  or more l-transitions
• We call this set of states L(q)
• Next, compute all states reachable from any
  element of L(q) using the character a
• We can denote these states as d(L(q),a)
• Finally, compute all states reachable from states
  in d(L(q),a) using 0 or more l-transitions
• We denote these states as L(d(L(q),a))
• This is the desired answer

151
Example

• d1(1,b) 3,4,5
• Compute L(1), all states reachable from state 1
  using 0 or more l-transitions
• L(1) 1,2
• Compute d(L(1),b), all states reachable from any
  element L(1) of using the character b
• d(L(1),b) d(1,2,b)
• d(1,b) U d(2,b)
• U 3 3
• Compute L(d(L(1),b)), all states reachable from
  states in d(L(1),b) using 0 or more l-transitions
• L(d(L(1),b)) L(3)
• 3,4,5

152
Comments

• For extra credit, you should be able to execute
  this algorithm
• Convert any NFA-l into an equivalent NFA.
• For extra credit, you should understand the idea
  behind this algorithm
• Why the transition function is computed the way
  it is
• Why A1 may need to include q1 in some cases
• You should understand the importance of this
  algorithm
• Compiler role again
• Use in combination with previous algorithm for
  converting any NFA into an equivalent FSA to
  create a new algorithm for converting any NFA-l
  into an equivalent FSA

153
LNFA-l LFSA

• Implications
• Let us primarily use the term LFSA to refer to
  this language class
• Given a language L is in LFSA
• We know there exists an FSA M s.t. L(M) L
• We know there exists an NFA M s.t. L(M) L
• To show a language L is in LFSA
• Show there exists an FSA M s.t. L(M) L
• Show there exists an NFA-l M s.t. L(M) L

154
Module 21

• Closure Properties for LFSA using NFAs
• From now on, when I say NFA, I mean any NFA
  including an NFA-l unless I add a specific
  restriction
• union (second proof)
• concatenation
• Kleene closure

155
LFSA closed under set union(again)
156
LFSA closed under set union

• Let L1 and L2 be arbitrary languages in LFSA
• Let M1 and M2 be NFAs s.t. L(M1) L1, L(M2)
  L2
• M1 and M2 exist by definition of L1 and L2 in
  LFSA and the fact that every FSA is an NFA
• Construct NFA M3 from NFAs M1 and M2
• Argue L(M3) L1 union L2
• There exists NFA M3 s.t. L(M3) L1 union L2
• L1 union L2 is in LFSA

157
Visualization

• Let L1 and L2 be arbitrary languages in LFSA
• Let M1 and M2 be NFAs s.t. L(M1) L1, L(M2)
  L2
• M1 and M2 exist by definition of L1 and L2 in
  LFSA and the fact that every FSA is an NFA
• Construct NFA M3 from NFAs M1 and M2
• Argue L(M3) L1 union L2
• There exists NFA M3 s.t. L(M3) L1 union L2
• L1 union L2 is in LFSA

LFSA
158
Algorithm Specification

• Input
• Two NFAs M1 and M2
• Output
• NFA M3 such that L(M3) ?

NFA M1 NFA M2
NFA M3
159
Use l-transition
a
M1
a,b
a,b
a,b
M2
160
General Case
161
Construction

• Input
• NFA M1 (Q1, S1, q1, d1, A1)
• NFA M2 (Q2, S2, q2, d2, A2)
• Output
• NFA M3 (Q3, S3, q3, d3, A3)
• What is Q3?
• Q3
• What is S3?
• S3 S1 S2
• What is q3?
• q3

162
Construction

• Input
• NFA M1 (Q1, S1, q1, d1, A1)
• NFA M2 (Q2, S2, q2, d2, A2)
• Output
• NFA M3 (Q3, S3, q3, d3, A3)
• What is A3?
• A3
• What is d3?
• d3

163
Comments

• You should be able to execute this algorithm
• You should understand the idea behind this
  algorithm
• You should understand how this algorithm can be
  used to simplify design
• You should be able to design new algorithms for
  new closure properties
• You should understand how this helps prove result
  that regular languages and LFSA are identical
• In particular, you should understand how this is
  used to construct an NFA M from a regular
  expression r s.t. L(M) L(r)
• To be seen later

164
LFSA closed under set concatenation
165
LFSA closed under set concatenation

• Let L1 and L2 be arbitrary languages in LFSA
• Let M1 and M2 be NFAs s.t. L(M1) L1, L(M2)
  L2
• M1 and M2 exist by definition of L1 and L2 in
  LFSA and the fact that every FSA is an NFA
• Construct NFA M3 from NFAs M1 and M2
• Argue L(M3) L1 concatenate L2
• There exists NFA M3 s.t. L(M3) L1 concatenate
  L2
• L1 concatenate L2 is in LFSA

166
Visualization

• Let L1 and L2 be arbitrary languages in LFSA
• Let M1 and M2 be NFAs s.t. L(M1) L1, L(M2)
  L2
• M1 and M2 exist by definition of L1 and L2 in
  LFSA and the fact that every FSA is an NFA
• Construct NFA M3 from NFAs M1 and M2
• Argue L(M3) L1 concatenate L2
• There exists NFA M3 s.t. L(M3) L1 concatenate
  L2
• L1 concatenate L2 is in LFSA

LFSA
167
Algorithm Specification

• Input
• Two NFAs M1 and M2
• Output
• NFA M3 such that L(M3)

NFA M1 NFA M2
NFA M3
168
Use l-transition
a
M1
a,b
a,b
a,b
M2
169
General Case
170
Construction

• Input
• NFA M1 (Q1, S1, q1, d1, A1)
• NFA M2 (Q2, S2, q2, d2, A2)
• Output
• NFA M3 (Q3, S3, q3, d3, A3)
• What is Q3?
• Q3
• What is S3?
• S3 S1 S2
• What is q3?
• q3

171
Construction

• Input
• NFA M1 (Q1, S1, q1, d1, A1)
• NFA M2 (Q2, S2, q2, d2, A2)
• Output
• NFA M3 (Q3, S3, q3, d3, A3)
• What is A3?
• A3
• What is d3?
• d3

172
Comments

• You should be able to execute this algorithm
• You should understand the idea behind this
  algorithm
• You should understand how this algorithm can be
  used to simplify design
• You should be able to design new algorith