Recursive descent parsing

1
Recursive descent parsing
2
Some notes on recursive descent

• The starter code that I gave you did not exactly
  fit the grammar that I gave you
• Both work, both are correct
• Many things can be coded either recursively or
  iteratively
• I gave you an iterative grammar and recursive code

3
Recursive rule for lttermgt

• lttermgt ltfactorgt ltmultiply_operatorgt lttermgt
• public boolean term() if (!factor()) return
  false if (!multiplyOperator()) return true
  if (!term()) error("No term after '' or '/'
  ") return true

4
Iterative rule for lttermgt

• lttermgt ltfactorgt ltmultiply_operatorgt
  ltfactorgt
• public boolean term() if (!factor()) return
  false while (multiplyOperator())
  if (!factor()) error("No factor after '' or '/'
  ") return true

5
Parse trees

• Every construct (statement, expression, etc.) in
  a programming language can be represented as a
  parse tree

6
Recognizers and parsers

• A recognizer tells whether a given string
  belongs to (is correctly described by) a
  grammar
• A parser uses a grammar to construct a parse tree
  from a given string
• One kind of parser is a recursive descent parser
• Recursive descent parsers have some
  disadvantages
• They are not as fast as some other methods
• It is difficult to provide really good error
  messages
• They cannot do parses that require arbitrarily
  long lookaheads
• And some advantages
• They are exceptionally simple
• They can be constructed from recognizers simply
  by doing some extra workspecifically, building a
  parse tree
• Recursive descent parsers are great for quick
  and dirty parsing jobs

7
The Stack

• One easy way to do recursive descent parsing is
  to have each parse method take the tokens it
  needs, build a parse tree, and put the parse tree
  on a global stack
• Write a parse method for each nonterminal in the
  grammar
• Each parse method should get the tokens it needs,
  and only those tokens
• Those tokens (usually) go on the stack
• Each parse method may call other parse methods,
  and expect those methods to leave their results
  on the stack
• Each (successful) parse method should leave one
  result on the stack

8
Example while statement

• ltwhile statementgt while ltconditiongt ltblockgt
• The parse method for a while statement
• Calls the Tokenizer, which returns a while
  token
• Makes the while into a tree node, which it puts
  on the stack
• Calls the parser for ltconditiongt, which parses a
  condition and puts it on the stack
• Stack now contains while ltconditiongt (top
  is on right)
• Calls the parser for ltblockgt, which parses a
  block and puts it on the stack
• Stack now contains while ltconditiongt
  ltblockgt
• Pops the top three things from the stack,
  assembles them into a tree, and pushes this tree
  onto the stack

9
Recognizing a while statement

• // ltwhile statementgt while ltconditiongt
  ltblockgtprivate boolean whileStatement() if
  (keyword("while")) if (condition())
  if (block()) return
  true else error("Missing '' ")
  else error("Missing ltconditiongt")
  return false

Why do you suppose I named this method
whileStatement() instead of while() ?
10
Parsing a while statement

• // ltwhile statementgt while ltconditiongt
  ltblockgtprivate boolean whileStatement() if
  (keyword("while")) if (condition())
  if (block())
  makeTree(3, 2, 1) return true
  else error("Missing '' ")
  else error("Missing ltconditiongt")
  return false

• This code assumes that keyword(String),
  condition(), and block() all leave their results
  on a stack
• On the stack, while 3, ltconditiongt 2, ltblockgt
  1

11
Alternative code

• public boolean whileStatement() if
  (keyword("while") condition() block())
  makeTree(3, 2, 1) return true
  return false
• No room for an error condition in this code

12
Alternative code with one message

• public boolean whileStatement() if
  (keyword("while")) if (condition())
  (block()) makeTree(3, 2, 1)
  return true
  error("Error in \"while\" statement")
  return false

13
Simple makeTree() method

• After recognizing a while loop, the stack looks
  like this

• And I could have written code like this
• private void makeTree() Tree right
  stack.pop() Tree left stack.pop()
  Tree root stack.pop() root.addChild(left)
  root.addChild(right) stack.push(root)

• This code assumes that the root is the third
  item down, etc., and that isnt always the
  case
• I found it more convenient to write more
  flexible methods

14
More general makeTree method

• private void makeTree(int keyword, int left, int
  right) Tree root getStackItem(keyword)
  Tree leftChild getStackItem(left) Tree
  rightChild getStackItem(right)
  stack.pop() stack.pop() stack.pop()
  root.addChild(leftChild)
  root.addChild(rightChild) stack.push(root)

15
Parser methods

• In the BNF, I have one long definition for
  ltcommandgt
• ltcommandgt ltmovegt ltexpressiongt
  lteolgt "turn" ltdirectiongt lteolgt
  "take" ltobjectgt lteolgt
  "drop" ltobjectgt lteolgt ...
• In my code, I broke that into multiple methods
• ltcommandgt ltmove commandgt ltturn
  commandgt lttake commandgt
  ltdrop commandgt ...

16
My command() method

• public boolean command() if (move())
  return true if (turn()) return true
  if (take()) return true if (drop())
  return true ... return false

17
Helper methods

• I wrote a number of helper methods for the Parser
  and for the ParserTest classes
• Its helpful to have methods to build trees
  quickly and easily
• private makeTree(String op)
• private Tree makeTree(String op, Tree left, Tree
  right)
• Java 5 allows a variable number of arguments, so
  you could writeprivate Tree makeTree(String op,
  Tree... children)
• Another useful method is assertStackTop, which is
  just
• private void assertStackTop(Tree expected)
  assertEquals(expected, parser.stack.peek())
• Example
• Tree condition makeTree("", "2",
  "2")assertStackTop(makeTree("if", condition,
  "list"))

18
Final comments

• You are welcome to use any of this code, but...
• ...my code is never the last word on anything!
• Code can always be improved
• While I think my code is generally useful, you
  always have to understand it well enough to adapt
  it to your particular needs

19
The End