[Haskell-beginners] Please help with this basic lexical analyzer program, implemented using the "Alex" tool

[Costello, Roger L.]

Hi Folks,

I am learning how to use the Haskell lexical analyzer tool called "Alex" [1].

I am trying to implement a lexical analyzer for this string (an email "From:" header):

	From: "John Doe" <john at doe.org>

I want to break it up into this list of tokens:

[
  From,
  DisplayName "John Doe",
  Email,
  LocalName "john",
  Domain "doe.org"
]

Below is my implementation. It works fine if the string doesn't contain a display name. That is, this works fine:

let s = "From: <john at doe.org>"
alexScanTokens s

However, when I include a display name, I get this error message:

[From*** Exception: lexical error

That is, this results in an error:

let s = "From: \"John Doe\" <john at doe.org>"
alexScanTokens s

I am guessing that this part of my "Alex" program is causing the error:

\"[a-zA-Z ]+\" 		{ \s -> DisplayName (init (tail s)) }

In Alex the left side is a regular expression:

	\"[a-zA-Z ]+\"

and the right side is the action to be taken when a string is found that matches the regular expression:

	{ \s -> DisplayName (init (tail s)) }

Any thoughts on what the problem might be?

Here is my lexical analyzer program:
-------------------------------------------------
{
module Main (main) where
}

%wrapper "basic"

$digit = 0-9			-- digits
$alpha = [a-zA-Z]		-- alphabetic characters

tokens :-

  $white+				;
  From:					{ \s -> From }
  \"[a-zA-Z ]+\" 				{ \s -> DisplayName (init (tail s)) }
  \<        					{ \s -> Email }
  [$alpha]+@				{ \s -> LocalPart (init s) }
  [$alpha\.]+>	   			{ \s -> Domain (init s) }

{
-- Each action has type :: String -> Token

-- The token type:
data Token =
	From 			|
	DisplayName String  	|
	Email			|
	LocalPart String	|
	Domain String		
	deriving (Eq,Show)

main = do
  s <- getContents
  print (alexScanTokens s)
}
-------------------------------------------------            
[1] The "Alex" lexical analyzer tool may be found at this URL: http://www.haskell.org/alex/doc/html/introduction.html