.NET Design Patterns

Lexical analysis

The lexical analyzer groups characters into tokens including '+', '-', '/', '*', SIN, COS, and so on. In the process, the module feeds the parser when a request is made to it. Rather than doing a lexical scan of the entire input, the parser requests the next token from the lexical analyzer. In our expression evaluator, the following tokens are returned by the lexical analyzer upon request for the next token by the parser:

    public enum TOKEN 
    { 
      ILLEGAL_TOKEN = -1, // Not a Token 
      TOK_PLUS = 1, // '+' 
      TOK_MUL, // '*' 
      TOK_DIV, // '/' 
      TOK_SUB, // '-' 
      TOK_OPAREN, // '(' 
      TOK_CPAREN, // ')' 
      TOK_DOUBLE, // '(' 
      TOK_TPARAM, // $t 
      TOK_SIN, // SIN 
      TOK_COS, // COS 
      TOK_NULL // End of string 
    } 

The lexical analyzer module scans through the input, ...