[Haskell-cafe] GHC, odd concurrency space leak

[Bertram Felgenhauer]

Bulat Ziganshin wrote:
> > This expands as
> 
> >     always a = a >> always a
> >              = a >> a >> always a
> >              = a >> a >> a >> always a
> >             ...
> > where each >> application is represented by a newly allocated object
> > (or several, I have not looked at it in detail) on the heap.
> 
> why you think so?

At the time I wrote this, because it explains the space leak and because
the space leak disappears if I address this precise issue. But I've
since verified the theory by inspecting Core and Cmm code.

> i always thought that >> in ghc just sequentially
> executes statements, the RealWorld magic exists only at compile-time

Yes, that's what happens once (>>) gets actually executed in IO. But
this fact and the RealWorld token have nothing to do with the whole
issue, which is about accumulating a chain of IO actions that have not
yet been executed.

I'll continue to write a >> b, which in IO, modulo newtypes, stands for

   \(s :: RealWorld#) -> case a s of (s', _) -> b s'

The fact that the state token disappears at runtime does not change
that this is a closure, represented by a (function) heap node.

So we have some IO action

    let x = always a

Now we run x, but also hold onto the corresponding thunk to reuse it
later, say

    let x = always a
    in  x >> x

In order to execute that, x is forced, and evaluated to

    let x = let x' = always a in a >> x'
    in  x >> x

or, equivalently,

    let x' = always a
        x  = a >> x'
    in  x >> x

Then the first step of the IO action is performed, resulting in

    let x' = always a
        x  = a >> x'
    in  x' >> x

And now the same reduction happens again for x',

    let x2 = always a
        x' = a >> x2
        x  = a >> x'
    in  x2 >> x

and then again for x2,

    let x3 = always a
        x2 = a >> x3
        x' = a >> x2
        x  = a >> x'
    in  x2 >> x

and so on, ad infinitum. This leaks memory because x, x', x2 etc. can't
be garbage collected - there's still a reference to x. Note that this
also explains why the space leak disappears if we remove the 'forever'
in the spawner thread in the original example.

This would not happen if the 'always a' was reused, i.e. if the code
tied a knot as

   let act = a >> act in act

does, but as you can see in the Core (and even Cmm if you look closely
enough) that does not happen in those cases where the code leaks memory.

HTH,

Bertram