Space leak due to pairs

[Simon Marlow]

Shin-Cheng Mu wrote:
> Hi,
> 
> I am trying to fix a space leak which I suspected to be
> caused by an old problem involving lazy pattern matching
> for pairs. P. Wadler and J. Sparud each suggested fixes
> to the compiler to resolve the space leak. I was wondering
> whether they are implemented in GHC. In the GHC Users
> mailing list archive I found a message in 2004 from Simon Peyton
> Jones saying that GHC does have the fix, but its effects
> are fragile when mixed with other optimisations:
> 
>   http://www.haskell.org/pipermail/glasgow-haskell-users/2004-August/ 
> 007023.html
> 
> I am wondering what the situation is now, and whether there
> is a way to make the fix work for my code.

The situation is still the same.  The optimisation is implemented, but 
doesn't work in all cases.  Fixing the general case seems difficult at 
least - we don't have a clear idea for how to do it.

>      *    *    *
> 
> The problematic code looks like:
> 
>   idX (StartEvent a : strm) =
>       let (ts, strm') = idX strm
>           (us, strm'') = idX strm'
>       in (StartEvent a : ts ++ EndEvent a : us, strm'')
> 
> The program processes a stream of XML events, and in this case
> it simply returns the stream itself with a minimal amount of
> validation. The intention of the code was to thread "strm"
> through the recursive calls and free the items in strm as soon
> as they are processed. So the program should use constant space
> in the heap (and stack space proportional to the depth of the
> XML tree). However, with this definition I needed about 8Mb of
> heap for a 1Mb input file.
 >
> The situation is the same even if I do not use -O or -O2.
> So it seems that the GC fix did not work for this case.
> Is there a way to make it work?
> 
> If I do have to alter the code to avoid the space leak,
> what is the best way to do it?

I'm sorry I can't help with specifics.  This kind of space leak usually 
requires deep insight and a lot of time staring at the code and heap 
profiles.  It's a big problem, I know.

Cheers,
	Simon