Fwd: Garbage collection

[Facundo Domínguez]

> So technically, your example might need to involve using g (and forceful GC at a certain point during execution)

Good observation.

> Maybe a stupid question, sorry: The RemoteTable generated using
> template-haskell in CH without XStaticPointers would keep CAFs alive. So the
> XStaticPointers extension does not entail using such a table?

That's correct. The extension is a substitute for the remote table. In
addition, it has the compiler do what remote tables demanded from the
user:
 * adding functions to the remote table before they are looked up,
 * collecting the table pieces from the various modules into a global table.

> Another question: Would it be sufficient to desugar "static g" to
>         g `seq` StaticPtr(StaticName "" "Main" "g")
> instead of introducing a stable ptr and all that?

This keeps g alive only while the expression is not evaluated to HNF.
The solution I proposed is flawed as well, since it relies on the
desugared static form being evaluated to HNF for the CAF to be
referenced with a StablePtr.

Anyway, after this much time we figured out how to implement the
static pointer table.

> Finally, there is a flag keepCAFs in the runtime which you can set to secure
> the CAFs for the entire run. The parallel runtimes for Eden and GUM (as well
> as my "packman" serialisation) do this.

Good to know about that.

Thank you,
Facundo


> AFter all, g is a CAF, so it is anyway "stable" in some sense, as long as it
> is alive.
>
> However, I conjecture that this only fixes the one-node test, not the actual
> use case (sending "static" stuff over the wire).
>
> Finally, there is a flag keepCAFs in the runtime which you can set to secure
> the CAFs for the entire run. The parallel runtimes for Eden and GUM (as well
> as my "packman" serialisation) do this.

Facundo

On Tue, Nov 18, 2014 at 1:20 PM, Jost Berthold
<berthold at mathematik.uni-marburg.de> wrote:
> Hi Facundo,
>
> You are completely right, the CAF named "g" might be accessed at any time
> during the program execution. Parallel Haskell systems with distributed heap
> (and runtime-supported serialisation) need to keep all CAFS alive for this
> reason.
>
> Some comments inline along your mail:
>
>>   While working in the StaticPointers language extension [1], we
>> found we have some unusual CAFs which can be accessed after some
>> periods of time where there is no reference to them.
>>
>>   For instance, the following program when compiled contains no
>> reference to `g`. `g` is actually looked up at runtime in symbol
>> tables via the call to `deRefStaticPtr`.
>> g :: String
>> g = "hello"
>>
>> main =
>>   deRefStaticPtr (static g) >>= putStrLn
>
>
> The bad scenario is certainly one where CAF g (a static thunk) is evaluated
> during execution (i.e. turned into an indirection into the heap), and then
> garbage-collected, as it might not be referenced by any (runnable) thread.
> This GC does not revert the indirection into a thunk. Why should it, there
> are no references to it, right? ;-)
>
> So technically, your example might need to involve using g (and forceful GC
> at a certain point during execution):
>
> main = putStrLn g >> performGC >>
>        deRefStaticPtr (static g) >>= putStrLn
>
>>
>> Desugars to:
>>
>> g :: String
>> g = "hello"
>>
>> main =
>
>     putStrLn g >> performGC >>
>>
>>   deRefStaticPtr (StaticPtr (StaticName "" "Main" "g")) >>= putStrLn
>
>
> During performGC, there would be no reference to g from any thread's stack.
> I am of course assuming that g is indeed a thunk, and not statically
> evaluated to a string during compilation (I am unsure whether GHC would do
> that).
>
>> In principle, there is nothing stopping the garbage collector from
>> reclaiming the closure of `g` before it is dynamically looked up.
>
>
> Maybe a stupid question, sorry: The RemoteTable generated using
> template-haskell in CH without XStaticPointers would keep CAFs alive. So the
> XStaticPointers extension does not entail using such a table?
>
>> We are considering using StablePtrs to preserve `g`. So the code
>> desugars instead to:
>>
>> g :: String
>> g = "hello"
>>
>> main =
>>   deRefStaticPtr (let x = StaticPtr (StaticName "" "Main" "g")
>>                          in unsafePerformIO $ newStablePtr g >> return x
>>                         ) >>= putStrLn
>>
>
> Another question: Would it be sufficient to desugar "static g" to
>         g `seq` StaticPtr(StaticName "" "Main" "g")
> instead of introducing a stable ptr and all that?
> AFter all, g is a CAF, so it is anyway "stable" in some sense, as long as it
> is alive.
>
> However, I conjecture that this only fixes the one-node test, not the actual
> use case (sending "static" stuff over the wire).
>
> Finally, there is a flag keepCAFs in the runtime which you can set to secure
> the CAFs for the entire run. The parallel runtimes for Eden and GUM (as well
> as my "packman" serialisation) do this.
>
> Yes, obviously, this opens a memory leak. It would be nice to not "keep" but
> "revert" the CAFs (ghci does that) but on a "by-need" basis when a simple GC
> cannot reclaim enough space; this would plug the mem.leak. This requires a
> modification to the GHC runtime system, and it is unclear _which_ CAFs to
> prefer when starting to revert. But I think it would be a more generally
> useful feature.
> However, this discussion (runtime/GC features) leads us straight out of the
> design goals of "-XStaticPointers", I guess...
>
> Best regards,
> Jost
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://www.haskell.org/mailman/listinfo/ghc-devs