Bug in GC's ordering of ForeignPtr finalization?

[Bertram Felgenhauer]

Dear Ben,

Ben Gamari wrote:
> After looking into this issue in a bit more depth, I'm even more
> confused. In fact, I would not be surprised if I have stumbled into a
> bug in the GC.
[...]
>         MessagesMessage
>               |   
>               |  msmpp
>               \/
>         QueryMessages
>               |
>               |  qmpp
>               \/
>             Query
> 
> As we can see, each MessagesMessage object in the Messages list
> resulting from queryMessages holds a reference to the Query object from
> which it originated. For this reason, I fail to see how it is possible
> that the RTS would attempt to free the Query before freeing the
> MessagesPtr.

When a garbage collection is performed, the RTS determines which heap
objects are still reachable. The rest is then freed _simultaneously_,
and the corresponding finalizers are run in some random order.

So assuming the application holds a reference to the MessagesMessage
object for a while and then drops it, the GC will detect unreachability
of all the three objects at the same time and in the end, the finalizer
for MessagesMessage may be run before that of Query.

So I think this is not a bug.

To solve this problem properly, libnotmuch should stop imposing order
constraints on when objects are freed - this would mean tracking
references using talloc_ref and talloc_unlink instead of
talloc_free inside the library.

For a bindings author who does not want to touch the library, the best
idea I have is to add a layer with the sole purpose of tracking those
implicit references.

Best regards,

Bertram