Failed OccName lookup in RnEnv with Template Haskell Hack

[Clemens Fruhwirth]

At Fri, 3 Nov 2006 09:51:50 -0000,
"Simon Peyton-Jones" <simonpj at microsoft.com> wrote:
> 
> | The problem is that the RdrNames carried in the HsExpr are not found
> | in the Renamer environment. To give you an example
> | 
> | foobar =
> |  $(return (InternalE (HsVar (mkUnqual varName
> | 				      (mkFastString "myfoo")))))
> | 
> | which should in the overall result "foobar = myfoo"
> | 
> | Running this gives me a "Not in scope" error but the myfoo variable is
> | obviously in the global environment as running this with
> | -ddump-rn-trace shows:
> 
> Well that is indeed puzzling, but you are doing something quite tricky
> here.  In TH, a TH splice uses only data types and functions declared in
> the TH library.  But your splices use data types declared in GHC itself.
> I'm not quite sure how you make the compilation and linking stuff work
> out, but you certainly need to take great care that the TH splice code
> is uses the same data types, compiled in the same way, as the compiler
> it's linked with.  Yes, it's hard to think about; I don't have it fully
> paged in myself.
> 
> So the ice is thin.  I can't account for the behaviour you are seeing --
> but I can't see how to help you more.  Perhaps do the HsVar and VarE
> thing in the *same* expression (in a pair, say), and confirm that they
> do generate the same HsExpr. Then start tracing the global envt lookup
> and its results.

Nothing is more effective in blocking you to find a solution than
broken a test case.

I was creating thousands of FastStrings in the template haskell
splice, I was even abusing conversion routings of the TH->HsSyn
converted to create fast strings for me, but all fast strings I could
ever possible create in a splice, (even with the help of the TH
converter) would have been interned in linker environment of the
splice! This is different from the compiler's linker environment. The
data structures created by the splice are relinked into the compiler
environment, but the FastStrings are incorrectly relinked. That is,
they are not interned in the compilers FastString symbol 
table. That's not the linkers fault of course, as it can not know that
creating a FastString has a side effect that must be redone in the
environment it's linking to.

I solved this problem partially for HsVar by recreating the
FastStrings it contains via back-and-forth conversion to String. Of
course, this is very cumbersome if I would have to do this for every
FastString/OccName/RdrName that is contained in an HsExpr tree.
Writting the tree traversal code is the exhausting part in this case,
but maybe generics will save us one day.

However, the best fix in my case is to lift the HsExpr generating
function into the compiler. All FastStrings are now created in the
proper environment.

Thanks for your suggestions, Simon.
--
Fruhwirth Clemens - http://clemens.endorphin.org 
for robots: sp4mtrap at endorphin.org