mallocForeignPtr vs. C

[Evan Laforge]

On Mon, Jul 12, 2010 at 12:54 PM, Edward Z. Yang <ezyang at mit.edu> wrote:
> Excerpts from Evan Laforge's message of Mon Jul 12 14:56:11 -0400 2010:
>> But I'm not convinced that's actually enough because the C code is
>> still running outside of a withForeignPtr.  I would have to do
>> something really hairy like call back to C from the haskell callback,
>> wrapping it in withForeignPtr.  Is there a better way to convince the
>> GC that the pointer should remain alive until I explicitly release it
>> from C?  The ideal I think would be a ref count that would keep the
>> pointer alive while >0, that could be decremented from C.
>
> The function you are looking for is withForeignPtr.  touchForeignPtr also
> works.

Well, what I'm worried about is that withForeignPtr says you should
only use the pointer from inside it.  The situation here is that I've
passed a pointer to C.  C wants to share ownership of the pointer, so
even if all haskell references are gone, it needs to stay alive until
C says so.  So there's nothing to pass to withForeignPtr.  C uses a
ptr, not a foreign ptr.

My proposed solution is to stash a reference to the foreign pointer
inside a callback in the hopes that will be treated as a GC root and
keep it alive manually, until the callback is deleted manually with
freeHaskellFunPtr.

So the awkward version would be:

- pass funptr haskell callback to C, that will return the desired ptr
- C wants to use the ptr, so it calls back to haskell, with then calls
back to C inside a withForeignPtr, passing the underlying ptr in
- now the uses of the ptr are dynamically inside of a withForeignPtr
- when C is done, it calls freeHaskellFunPtr on the funptr

It makes the C awkward and you wind up with a stack that looks like
haskell -> C -> haskell -> C.  However, my suggested abbreviation is:

- pass ptr to C directly, along with a funptr haskell callback to keep
the ptr alive
- C just uses the ptr directly, trusting that the funptr will keep it alive
- when it's done, free the funptr as normal

But where I'm unclear is what the funptr has to do to keep the ptr
alive.  Return it?  Call touchForeignPtr on it?  It seems weird to me
because this funptr is never called, it's just there for the GC.

Another hack I thought of:

- store ptr in a global MVar in haskell, pass it to C
- C does its thing, ptr stays alive because haskell still has a reference
- now when C calls the finalize funptr, it deletes the ptr from the
haskell MVar, which releases haskell's hold on it

Effectively, this is using the global MVar as a GC root instead of the
funptr itself.  I'm much more confident that the GC is going to follow
module level CAFs than some random funptrs allocated with a foreign
"wrapper" call.  However, the "wrapper" calls must serve as GC roots
too, because otherwise what guarantees that variables captured in its
closure stay alive?  Right?