Fwd: Garbage collection
Jost Berthold
berthold at Mathematik.Uni-Marburg.de
Tue Nov 18 15:20:57 UTC 2014
Hi Facundo,
You are completely right, the CAF named "g" might be accessed at any
time during the program execution. Parallel Haskell systems with
distributed heap (and runtime-supported serialisation) need to keep all
CAFS alive for this reason.
Some comments inline along your mail:
> While working in the StaticPointers language extension [1], we
> found we have some unusual CAFs which can be accessed after some
> periods of time where there is no reference to them.
>
> For instance, the following program when compiled contains no
> reference to `g`. `g` is actually looked up at runtime in symbol
> tables via the call to `deRefStaticPtr`.
> g :: String
> g = "hello"
>
> main =
> deRefStaticPtr (static g) >>= putStrLn
The bad scenario is certainly one where CAF g (a static thunk) is
evaluated during execution (i.e. turned into an indirection into the
heap), and then garbage-collected, as it might not be referenced by any
(runnable) thread.
This GC does not revert the indirection into a thunk. Why should it,
there are no references to it, right? ;-)
So technically, your example might need to involve using g (and forceful
GC at a certain point during execution):
main = putStrLn g >> performGC >>
deRefStaticPtr (static g) >>= putStrLn
> Desugars to:
>
> g :: String
> g = "hello"
>
> main =
putStrLn g >> performGC >>
> deRefStaticPtr (StaticPtr (StaticName "" "Main" "g")) >>= putStrLn
During performGC, there would be no reference to g from any thread's stack.
I am of course assuming that g is indeed a thunk, and not statically
evaluated to a string during compilation (I am unsure whether GHC would
do that).
> In principle, there is nothing stopping the garbage collector from
> reclaiming the closure of `g` before it is dynamically looked up.
Maybe a stupid question, sorry: The RemoteTable generated using
template-haskell in CH without XStaticPointers would keep CAFs alive. So
the XStaticPointers extension does not entail using such a table?
> We are considering using StablePtrs to preserve `g`. So the code
> desugars instead to:
>
> g :: String
> g = "hello"
>
> main =
> deRefStaticPtr (let x = StaticPtr (StaticName "" "Main" "g")
> in unsafePerformIO $ newStablePtr g >> return x
> ) >>= putStrLn
>
Another question: Would it be sufficient to desugar "static g" to
g `seq` StaticPtr(StaticName "" "Main" "g")
instead of introducing a stable ptr and all that?
AFter all, g is a CAF, so it is anyway "stable" in some sense, as long
as it is alive.
However, I conjecture that this only fixes the one-node test, not the
actual use case (sending "static" stuff over the wire).
Finally, there is a flag keepCAFs in the runtime which you can set to
secure the CAFs for the entire run. The parallel runtimes for Eden and
GUM (as well as my "packman" serialisation) do this.
Yes, obviously, this opens a memory leak. It would be nice to not "keep"
but "revert" the CAFs (ghci does that) but on a "by-need" basis when a
simple GC cannot reclaim enough space; this would plug the mem.leak.
This requires a modification to the GHC runtime system, and it is
unclear _which_ CAFs to prefer when starting to revert. But I think it
would be a more generally useful feature.
However, this discussion (runtime/GC features) leads us straight out of
the design goals of "-XStaticPointers", I guess...
Best regards,
Jost
More information about the ghc-devs
mailing list