[Haskell-cafe] Re: Case-insensitive lexing with Alex

[Geoffrey Mainland]

Joel Reymont wrote:
> 
> On Jan 28, 2008, at 2:44 PM, Geoffrey Mainland wrote:
> 
>> map toLower onto your input before you pass it to your lexer? Or do you
>> only want keywords to be case-insensitive?
> 
> 
> Just keywords. You can have "Array" or "array" or "aRrAy".

One old trick for reducing the size of lexers is to not put keyword
recognition into the lexer rules (that is, don't write rules to match
keywords explicitly), but to look up identifiers in a map from strings
to keyword tokens. In your case, apply toLower to the identifier before
you look it up in the map from keywords to tokens.

The C parsing library I gave you a link to off-list does its lexing
using this technique if you want a concrete example.

Geoff