Compiler Designs and Constructions Page 83 92

1
Compiler Designs and Constructions (Page 83 92)

• Chapter 4 Scanner (Lexical Analyzer)
• Objectives
• Introduction to Scanner
• Token Definition
• Types of Scanners
• Dr. Mohsen Chitsaz

2
Scanner (Lexical Analyzer)

• Task

GetNextToken
Scanner
Parser
Token
Symbol Table
3
Scanner

• Token
• Input File
• Keywords
• User id
• Symbols
• Numbers
• Preparation
• Remove Comments
• Case Conversion
• White Spaces
• Compiler Directives

(Token Continued)
4
Scanner

• Preparation of the output listing
• Create a listing file
• Keep line numbers
• Insert errors, warnings
• Table Summary

5
Scanner

• Definition
• Total, Sum, LongNum, Max
• If
• Input
• Disk -gt OS Buffer -gt File Buffer -gt Program
  Buffer
• Reading large blocks
• Min Disk access
• Unite allocations
• Lookahead/ Pushback

6
Scanner

• Buffering
• Buffer Size
• Max lexeme length

7
Scanner

• Lexeme A, NumOfBooks, Sum
• Token ID

8
Scanner

• Buffer Markers
• Start
• End
• Logical End of Buffer
• Next Input
• Previous Lexeme
• Beginning of Current Lexeme
• End of Current Lexeme
• Danger Zone
• Lookahead Characters