Haskell-src-ext

[Claus Reinke]

> ..Comments are simply
> discarded, and to be honest I really don't see how they could be kept
> in general, except in specific pre-defined places (like for Haddock).
> I'm sure you have ideas on that though. Layout is also not preserved
> exactly, and do { ... ; ... } results in the same AST as if done with
> layout instead of { ; }.

just recalling some previous discussions:

a) add a whitespace&layout field to every AST construct

    + should work well enough for source location info and
        for braces yes/no flags
    - doesn't work so well for comments: do they belong to
        the next or the previous item, or are they standalone text?

b) add a comment construct to the AST

    + avoids association issue, comments stand on their own
    - AST traversals have to deal with comments
    - AST transformations still have to decide which comments
        to move with which AST fragments
    + comments in untransformed code remain

c) add source location and layout info to every AST construct,
    keep comments out of the AST, link source locations to
    comment sections
    
    + keeps AST traversals simple
    - AST transformations still need to decide which comments 
        to move with which AST fragments
    + allows finding related comments after AST transformations
        (the ghc ticket suggests two lookup functions, for comments
        before/after a given AST fragment)
    + comments in untransformed code remain

d) ..? i'd be interested in other alternatives

so far, it seems that c is the way to go (HaRe used the 
Programatica's token stream as comment storage, with every
identifier having source location info, and some heuristics for
matching comments to AST fragments), though i'd be interested 
in other alternatives - keeping the parts of a split-up source
representation in sync during transformation and weaving 
everything back together before presentation proved tedious.

> Indeed, and I'm afraid haskell-src-exts will join the crowd in that
> regard. But to be honest I'm not sure haskell-src-exts *should* do
> those things you ask for, since the added machinery would be rather
> heavy-weight for applications that just want the basic stuff, which I
> guess is the vast majority of applications.

would the added machinery really affect applications that don't
care about comments/layout? with (c), there'd be one field not
to look at and a comment store to ignore.

and the bonus would be to enable a large variety of applications
based on source to source transformations (refactoring, pretty
printing, desugaring, layout<->braces, optimizations, ..).

but perhaps that isn't the target for haskell-src-ext, and things
like HaRe are currently hoping/aiming for the ghc api.
 
>>  ps it is good to hear that src-ext is supported, follows language
>> developments, and is separated from the other parts of your
>> projects!-)
> 
> Thanks, and yes I try to keep them as separate as possible, 
> knowing their usefulness to others.

keep up the good work!-) 

claus