| | |
Summary: Program 4 Lexical Analysis and Static Analysis (20 points)
Overview
Lexical analysis only breaks an input file (a program) into tokens. The job of building a symbol table and
deciding if the statements are correctly formed belongs to the parser. The lexical analyzer does not
have the sophistication to perform the complex tasks of the parser.
For this assignment you are to write a Ruby program to perform lexical analysis:
1. Produce a stream of token IDs, token Names, and (if appropriate) lexemes (the actual string
matched) associated with a Ruby input file.
2. Identify any errors by saying "Error on line X" where X is the input line number, and then stopping
execution. Basically, your lexer need only correctly identify that there is an error on line X not identify
what the error is. An error is anything that does not match something valid.
3. Produce the following static analysis
a) Average number of tokens per line
b) Token with most number of uses
c) Maximum number of tokens per line
|
