Simplifications of ContextFree Grammars

1
Simplifications of Context-Free Grammars

2
A Substitution Rule
Equivalent grammar
Substitute
3
A Substitution Rule
Substitute
Equivalent grammar
4
In general
Substitute
equivalent grammar
5
Nullable Variables
Nullable Variable
6
Removing Nullable Variables
Example Grammar
Nullable variable
7
Final Grammar
Substitute
8
Unit-Productions
Unit Production
(a single variable in both sides)
9
Removing Unit Productions
Observation
Is removed immediately
10
Example Grammar
11
Substitute
12
Remove
13
Substitute
14
Remove repeated productions
Final grammar
15
Useless Productions
16
Another grammar
Not reachable from S
17
contains only terminals
In general
if
then variable is useful
otherwise, variable is useless
18
A production is useless if any of
its variables is useless
19
Removing Useless Productions
Example Grammar
20
First
find all variables that can produce strings with
only terminals
Round 1
Round 2
21
Keep only the variables that produce terminal
symbols
(the rest variables are useless)
Remove useless productions
22
Second
Find all variables reachable from
Use a Dependency Graph
not reachable
23
Keep only the variables reachable from S
(the rest variables are useless)
Final Grammar
Remove useless productions
24
Removing All

• Step 1 Remove Nullable Variables
• Step 2 Remove Unit-Productions
• Step 3 Remove Useless Variables

25
Normal FormsforContext-free Grammars

26
Chomsky Normal Form
Each productions has form
or
variable
variable
terminal
27
Examples
Chomsky Normal Form
Not Chomsky Normal Form
28
Convertion to Chomsky Normal Form

• Example

Not Chomsky Normal Form
29
Introduce variables for terminals
30

Introduce intermediate variable
31
Introduce intermediate variable
32
Final grammar in Chomsky Normal Form
Initial grammar
33
In general
From any context-free grammar (which doesnt
produce ) not in Chomsky Normal Form
we can obtain An equivalent grammar
in Chomsky Normal Form
34
The Procedure
First remove Nullable variables Unit
productions
35
Then, for every symbol
Add production
In productions replace with
New variable
36
Replace any production
with
New intermediate variables
37
Theorem
For any context-free grammar (which doesnt
produce ) there is an equivalent grammar in
Chomsky Normal Form
38
Observations

• Chomsky normal forms are good
• for parsing and proving theorems

• It is very easy to find the Chomsky normal
• form for any context-free grammar

39
Greinbach Normal Form
All productions have form
symbol
variables
40
Observations

• Greinbach normal forms are very good
• for parsing

• It is hard to find the Greinbach normal
• form of any context-free grammar

41
Compilers

42
Machine Code
Program
Add v,v,0 cmp v,5 jmplt ELSE THEN add x,
12,v ELSE WHILE cmp x,3 ...
v 5 if (vgt5) x 12 v while (x !3)
x x - 3 v 10 ......
Compiler
43
Compiler
Lexical analyzer
parser
input
output
machine code
program
44
A parser knows the grammar of the programming
language
45
Parser
PROGRAM STMT_LIST STMT_LIST STMT
STMT_LIST STMT STMT EXPR IF_STMT
WHILE_STMT
STMT_LIST EXPR EXPR EXPR EXPR - EXPR
ID IF_STMT if (EXPR) then STMT
if (EXPR) then STMT else STMT WHILE_STMT
while (EXPR) do STMT
46
The parser finds the derivation of a particular
input
derivation
Parser
input
E gt E E gt E E E gt 10 EE gt
10 2 E gt 10 2 5
E -gt E E E E INT
10 2 5
47
derivation tree
derivation
E
E gt E E gt E E E gt 10 EE gt
10 2 E gt 10 2 5

E
E
10
E
E

2
5
48
derivation tree
E
machine code

E
E
mult a, 2, 5 add b, 10, a
10
E
E

2
5
49
Parsing

50
Parser
input string
derivation
grammar
51
Example
Parser
derivation
input
?
52
Exhaustive Search
Phase 1
Find derivation of
All possible derivations of length 1
53
(No Transcript)
54
Phase 2
Phase 1
55
Phase 2
Phase 3
56
Final result of exhaustive search
(top-down parsing)
Parser
input
derivation
57
Time complexity of exhaustive search
Suppose there are no productions of the form
Number of phases for string approx. w

58
For grammar with rules
Time for phase 1
possible derivations
59
Time for phase 2
possible derivations
60
Time for phase w is 2w
A total of 2w possible derivations
61
Total time needed for string
phase 1
phase w
phase 2
Extremely bad!!!
62
For general context-free grammars
There exists a parsing algorithm that parses a
string in time
The CYK parser