Calculator Example

1
Calculator Example

• Goals
• Show regular expression and BNF grammar rule
  examples for lexical and syntactic analysis.
• Show Unified Modeling Language graphic notations
  for composite data structures and for syntactic
  constraints on the parse and evaluation trees of
  well-formed expressions.

2
Structure of BNF Rules (Right-Recursive for
Nested-FSM Parser)

• LeftHand Side RightHand Side Rule
• (LHS) (RHS)
• ltcmdgt ltexprgt 1
• ltexprgt lttermgt 2 lttermgt ( -)
  ltexprgt 3
• lttermgt ltfactorgt 4 ltfactorgt (
  /) lttermgt 5
• ltfactorgt ltintgt 6 ltidentgt 7
• Grammar ltidentgt ltexprgt 8 Variables
  - ltfactorgt 9 ( ltexprgt ) 10

99s204Calculator.ppt - RJL rev. 990214- 2
3
Integer Calculator - LL Grammar (Right-Recursive
- for Nested-FSM Parser)

• ltcmdgt --gt ltexprgt
• ltexprgt --gt lttermgt lttermgt ( -)
  ltexprgt
• lttermgt --gt ltfactorgt ltfactorgt ( /)
  lttermgt
• ltfactorgt --gt ltintgt ltidentgt
  ltidentgt ltexprgt - ltfactorgt
  ( ltexprgt )
• _______
• Imbedded or essential recursion - this
  requires a push-down recognizer (a set of
  recursively callable FSRecognizers).

Right or tail recursion is replaced by
iteration in the syntax diagram on the next slide)
99s204Calculator.ppt - RJL rev. 990214- 3
4
Syntax Diagrams for BNF (1)
(A useful tool to visualize all syntactically
legal cases)
Expr
Term
/-
Term
True or Pass Exit
Possible False, Error or Fail Exits
Regular Expression represented by this syntax
diagram Expr Term Term Term -
Term or Expr Term ( -) Term
Valid Semigroup algebraic operations
include string concatenation product, alternate
choice addition, and for repetition, along
with parentheses for grouping.
5
Syntax Diagrams for BNF (2)
Expr
Term
/-
Term
True or Pass Exit
Factor
Term
)
Expr
(
Term
/-
Expr
Id

6
Syntax Diagrams for BNF (3)
Expr
Term
/-
Term
True or Pass Exit
Term
Factor
Fail-exits (NOT shown), are essential to
complete the design by handling illegal syntax
that does NOT calculate.
)
Expr
(
Term
/-
Expr
Id

7
Grammar with Extended BNF

• ltcmdgt --gt ltexprgt
• ltexprgt --gt lttermgt lttermgt ( -)
  lttermgt...
• lttermgt --gt ltfactorgt ltfactorgt ( /)
  ltfactorgt...
• ltfactorgt --gt ltintgt ltidentgt
  ltidentgt ltexprgt - ltfactorgt
  ( ltexprgt )
• ______
• (equivalent for parsing only - translation may
  require
• different semantic actions for first and/or
  last items.)

(... denotes iteration)
Indirect rightmost recursion Essential (embedde
d) recursion

99s204Calculator.ppt - RJL rev. 991118- 7
8
Integer Calculator - LR Grammar(left-recursive,
for yacc)

• ltcmdgt --gt ltexprgt
• ltexprgt --gt lttermgt ltexprgt ( -)
  lttermgt
• lttermgt --gt ltfactorgt lttermgt ( /)
  ltfactorgt
• ltfactorgt --gt ltintgt ltidentgt
  ltidentgt ltexprgt - ltfactorgt
  ( ltexprgt )
• ltidentgt --gt ltlettergt(ltlettergtltdigitgt)
• ltintgt --gt ltdigitgt ltdigitgt
• (BNF Rules must replace the operator by
  recursion.)

99s204Calculator.ppt - RJL rev. 991118- 8
9
Parse Tree Example
expr (1(y))(zltlt4-1)/2 term term
/ factor term factor int
( expr ) ( expr ) 2 expr
term expr - term term factor
term factor factor ( expr ) term
ltlt factor int int term id
int 1 1 factor z 4
factor id y

99s204Calculator.ppt - RJL rev. 991118- 9
10
Conversion to Abstract Syntax
expr (1(y))(zltlt4-1)/2 /
factor factor int
- 2 expr term expr
term term factor
ltlt factor factor expr term
factor int int term id
int 1 1 z 4 facto
r id y
(Operators promoted, operand productions not yet
compressed.)

99s204Calculator.ppt - RJL rev. 991118- 10
11
Abstract Syntax Evaluation Tree
expr (1(y)) (zltlt4-1) / 2
/ 2
- 1 1
y ltlt z
4
.


99s204Calculator.ppt - RJL rev. 991118- 11
12
UML Class Diagrams

• Class diagrams primarily summarize inheritance
  relationships among parse tree classes.
• Details are usually defined in interface (.h)
  files, or individual class diagrams.

Superclass or base class
Discriminator to identify the superclass (if
you must put it elsewhere, on a complex
diagram).
Binop
Subclasses or derived classes
Plus
Minus
13
Calculator UML Class Diagram Parse Tree Nodes
Node
Binop
Id
Int
Nament
Uminus
Plus
Minus
This is a summary overview diagram. It does not
show public members or the private links that
relate instances to each other. These details
are defined in interface (.h) files, or in
individual class diagrams. In either case,
these details should be hyperlinked to this
overview. Source node.hh, the header file for
calculator.cc.
Times
Div
Assign


99s204Calculator.ppt - RJL rev. 010131 - 13
14
Extended Class Diagram(for Abstract Syntax Tree)
Node
(Abstract root or base class)
Classes representing grammar variables have
components, which can be declared as reference
types.
Int
Binop
Id
Unop
Nament
Lshft
Plus
Minus
Uminus
PreDec
PreInc
Rshft
Div
Times
Assign


Operators marked with an asterisk are new in
99s204 project 2
99s204Calculator.ppt - RJL rev. 991118- 14
15
Class Decomposition Diagram - 1
Note C supports dynamic identification of and
casts to subclasses.
Scope Block
ProgHdr
level
11
11
11
11
11
11
0M
Statement or Expr
Argv
1N
11
Nament name
01
Term
11
AddOp type value tmploc
01
11
01
MulOp type value tmploc
Factor
FltVar value
IntVar value
01
11
11
11
1M
1M
0M
01
01
01
0M
01

IntLit
FltId
FltLit
IntId
UnyOpn
NestedScope
RptOpn
99s204Calculator.ppt - RJL rev. 990510 - 15
16
Class Decomposition Diagram - 2
Each rectangle represents one class of objects.
Arrows represent 1to1 or 1toM is-part-of
relations. UnyOpn has one Factor component
BinOpn has 2.
Scope Block scopeLevel
ProgHdr
level
11
11
11
11
0M
0N
Statement or Expr
Nament name value
Argv
01
01
01
01
0M
BinOpn operator retType retValue tmpLoc
0M
Compound Stmt/Block Subtype Repeat or newScope)
UnyOpn operator retType retValue tmpLoc
Literal type value
Id entry


99s204Calculator.ppt - RJL rev. 990510 - 16
17
Class Decomposition Diagram - 3
Compound block subtypes NewScope and Repeat (not
shown) contain one or two nested Stmt lists,
respectively. Requires type checking of operands
and type constraints on compound statement
components.
Scope Block scopeLevel
ProgHdr
level
11
11
11
11
0M
0N
Statement or Expr
Nament name value
Argv
01
01
01
01
0M
BinOpn operator retType retValue tmpLoc
0M
Compound Stmt/Block Subtype Repeat or newScope)
UnyOpn operator retType retValue tmpLoc
Literal type value
Id entry


99s204Calculator.ppt - RJL rev. 990510 - 17
18
Class Diagram with Aggregates
The directed arrows are 1M or 11
relations, depending on multiplicity.
Node
SymtabContainer
11
11
11
11
11
01
0M
0M
01
Binop Node l, r
Nament char name int value
Sym tab
01
Int value
Unop Node operand
0M
Id Nament symtab, entry
11
entry
0M
Lshft
Plus
Minus
Uminus
PreDec
PreInc
Rshft
Div
Times
Assign (Node l is_a Id)


(For , -- operand is_a Id )
99s204Calculator.ppt - RJL rev. 990510- 18
19
Testing Finite-State Recognizers

• Goal generate test cases for every grammar rule!
• It is not enough to test casually, or only once.
• It is not enough to test with only good input.
• Can test strings for FSRs be generated
  randomly?
• Yes (e. g., the cases can be generated by a
  random walk along FSRecognizer machine paths).
• Is there an equivalent random walk over the
  rules?
• Yes (provided that we pick the right Regular
  Expression grammar).
• Is this relevant to context-free recognizers?
• Yes - see later.

99s204Calculator.ppt - RJL rev. 991118- 19
20
Testing Lexical Analyzers

• Lexical analyzers have actions (side effects).
• Translators analyzers whose actions emit
  outputs.
• Use control state context to specify explicit
  sequential constraints. For example
• -?0-9 ,0-9 (a list of
  comma-separated integers).
• Use local variables to save (and later test)
  extra data state for non-regular languages.
  For example
• Counters to check string length constrains
• Stacks to check nested parentheses constraints
• Symbol tables to check for variable def before
  use.
• Side effects require more complex testing.

99s204Calculator.ppt - RJL rev. 991118- 20
21
Generating Test Strings

• Goal supply test data strings for each path
  through a DFSR and its State Transition Diagram
  or Table
• All test strings begin at the Start State (STT
  row).
• Good strings end at accepting (final) state
  rows.
• Bad strings end at rejecting (trap) state rows.
• Make sure you know which is which! (bad strings
  should NOT produce apparently good answers).
• Make sure your test cases traverse all possible
  paths or trajectories though states and state
  transitions.
• Good tools help by keeping track for you.
• Build a test case library that grow with the
  product.

99s204Calculator.ppt - RJL rev. 991118- 21
22
State Transition Tables

• Every DFSR has a State Transition Table (STT)
• One row per State s
• One column per Input symbol I
• Cells define the Next-State function s
  NS(s,I).
• Equivalent states have identical rows.
• For test case generation, add a new dead or
  trapping state row if necessary, and fill blank
  cells with this next state. Append T to Trap
  State, F to Final (Accept) State.
• Example STT for this STD

01
1
S0
S1
State Input 0 1 S0 S2 S1
S1T S1 S1 S2F S2 S1
0
1

S2
0
99s204Calculator.ppt - RJL rev. 990214- 22
23
Random Walks thru an STD ( 1)

• Begin in the starting state S0, with empty string
  .
• Mark this string as accepted (iff S0 is Final) or
  rejected (S0 is not an accepting or final state).
• Randomly choose outgoing transitions. Define a
  new string with this transitions label and add
  it to your list, marked accepted or rejected
  depending on the next states property.
• Repeat from last state, for each string not
  trapped.

Accepted 0(100) 00(100) Rejected
(001)1(01) Good strings 0, 00, 010, 000,
0010, 00010, 00100,.. Bad strings 1, 10, 11,
011, 0011,..
1
01
S0
S1
0
1

S2
0
99s204Calculator.ppt - RJL rev. 990214- 23
24
Testing Context-free Parsers

• Required when grammar contains nonlinear
  productions (more than one variable on RHS of a
  rule) and when grammar contains embedded or
  essential recursion.
• This requires a push-down recognizer
• For test generation, this is equivalent to a set
  of recursively callable finite-state recognizers.
• Syntax diagrams can define test cases
• (in context based on First and Follow sets).

25
Testing Calculator.cc

• Test cases regular expressions over the grammar
  paths through the syntax diagram.
• Examples (Let E expr, T term, F factor)
• E--gt T ((-)T)
• T --gt F ((/) F)
• F --gt (Id Int (E) Id E -F)
• ( \t is allowed anywhere outside of Int or Id.)
• A good input is any string obeying these rules.
• A bad string is any string not obeying these
  rules.
• Syntax Diagrams are good ways to see bad ones
  (Any appended illegal char should be detected).

99s204Calculator.ppt - RJL rev. 991118- 25
26
Calculator Syntax Diagrams
(A useful basis for selecting good and bad test
cases)
E --gt F /- F F
--gt Int ( E ) - F Id E
(Machine E calls machine
F below)
Fail-exits, which are NOT shown, are
candidates for bad test data (illegal syntax
that should NOT calculate).

99s204Calculator.ppt - RJL rev. 990214- 26
27
A Repeatable Testing Process

• Create two files input and expected output e.g.,
  ctest1.inp, ctest1.req (c is for calc).
• Cvs checkin a new revision of these files
  whenever they are modified or extended.
• Routinely add new test cases as needed for
  future re-use.
• Create a command (.sh/.bat) file to automate
  testing. Check it into cvs as well.

99s204Calculator.ppt - RJL rev. 990214- 27
28
Calculator Test Driver

• A command script (with ctest1 passed as its
  argument) should automate testing, as follows
• 1. Run calc on ctest1.in generate ctest1.out
  e.g., calc lt ctest1.in gt ctest1.out
• 2. Compare results to requirements e.g., run
  diff -b ctest1.out ctest1.req gt ctest1.dif
• 3. Run wc ctest1.dif if (wc gt 0) print
  ctest1.dif.

99s204Calculator.ppt - RJL rev. 990214- 28
29
Calculator - good and bad Tests
Example Inputs to E Good Bad F
F.. F-/F F-/F..
E --gt F /- F F
--gt Int ( E ) - F E
(Machine E calls machine F below)
Example Inputs to F Good Bad Int
dl-(... (E) dd... -F
-dl-(... Id Iddl-(.. (Id)E
Id-/F.. (repeat with whitespace)
Rev. 991014 to (Id )
Id
__________ For Bad inputs, select illegal
first-symbol or successor-symbol values.
99s204Calculator.ppt - RJL rev. 990214- 29
30
Examples of Bad Input Test Strings

• E ltexpressiongt, F ltfactorgt, E (F) is any
  string not an E (not an F).
• E -- gt F-/F.. (1)x -1(y) a)(1 123a
• E --gt F-/F.. 123) 1231 abc-1 abc--x
• E --gt F (below)
• F --gt dl-( )1 )-a a z
• F --gt ( E) (.) (12) () (1a)
• F --gt (E )... (12 (ab/) (1/2/3) (x2/z()
• F --gt dd 1a 2345z 12(3) 12345678999z
• F --gt -F -1a -)z -(3) -1
• F --gt Iddl--/F.. ZYx(1) Z(a) zy(1a)
  1zy
• F --gt Id(dl- zyxw) z1 zx) x(1x)
• F --gt Id-/F.. Z) z-1 zy//1 z---1

99s204Calculator.ppt - RJL rev. 990214- 30
31
Sample Bad Input Test Data
Four strings per line - 40 test strings in
all. (This data is in file CASE/99s204/ctest1.in
as forty single-line records.)

• E -- gt (1)x -1(y) a)(1 123a
• E --gt 123) 1231 abc-1 abc--x
• E --gt )1 )-a a z
• E --gt (.) (12) () (1a)
• E --gt (12 (ab/) (1/2/3) (x2/z()
• E --gt 1a 2345z 12(3) 12345678999z
• E --gt -1a -)z -(3) -1
• E --gt ZYx(1) Z(a) zy(1a) 1zy
• E --gt zyxw) z1 zx) x(1x)
• E --gt Z) z-1 zy//1 z---1

99s204Calculator.ppt - RJL rev. 990214- 31