[Haskell-cafe] Profiling makes memory leak go away? Is Haskell apractical language?

[Claus Reinke]

> reply = parse ... -- Lazily evaluated
> tokens = rTokens reply -- Has some values "immediately"
> list = D.toList tokens -- Has some values "immediately"
> mapM_ list print -- Start printing "immediately"!
.. 
> reply = parse ... -- Lazily evaluated
> result = rResult reply -- Lazy; has value when parsing is done
> extra = case result ... -- Lazy; has value when parsing is done
> parsed = rTokens reply -- Has some values "immediately"
> tokens = D.append parsed extra -- Has some values "immediately"
> list = D.toList tokens -- Has some values "immediately"
> mapM_ list print -- Starts printing "immediately"!
> 
> This *still* starts printing tokens immediately. However, while in the
> previous case the GC is smart enough to keep the program in constant
> memory size, in the second case it for some reasons starts missing more
> and more PAP objects so memory usage goes through the roof.

is that a nail for this hammer, perhaps?-)

    http://hackage.haskell.org/trac/ghc/ticket/917

if you don't use rResult reply, reply can be used and freed as it is used.
if you do use rResult reply, you are going to use it late, and that is going to hang 
on to reply, which is being expanded by the main thread of activities (rTokens).

i'm just guessing here, but if that is indeed the problem, you would need to exert
more control over what is evaluated when and shared where:

- evaluate rResult synchronously with rTokens, instead of rResult long after
    rTokens has unfolded the reply
- evaluate rResult independent of rTokens, on a separate copy of reply

since you want to use parts of the output before you can be sure whether the
whole input is correct, you might also want local errors instead of global ones
(i've seen a correct chunk of input, here is the corresponding chunk of output;
instead of here is a list of output chunks i've produced so far, i'll tell you later 
whether they are worth anything or whether they were based on invalid input).

> You'd think... but the fact of the matter is that while the first
> version works fine, the second doesn't, UNLESS I add the magic SCC
> section:
> 
>  extra = {-# SCC "magic" #-} case result ...
> 
> And compile with '-prof' (no '-O' flags). Then it somehow, finally, "get
> the idea" and the program runs perfectly well with constant memory
> consumption. Which, as you aptly put it, is very "fishy" indeed...

adding profiling might (another wild guess here..) lose sharing, just as in the ticket,
i used \()->[..] to avoid sharing of the list. (although that guess wouldn't necessarily
suggest this particular SCC to be useful, so perhaps it is the wrong track..)

hth,
claus