Finalizers strike back

[George Russell]

Alastair wrote
[snip]
> More importantly though, this does nothing at all to guarantee
> atomicity of Haskell code that manipulates global variables.  Consider
> a data structure consisting of a list of objects.  The main thread
> might add to this list and do searches in the list and the finalizer
> removes objects from the list:
> 
>   type State = IORef [Object]
> 
>   -- used by main
>   newObject :: Object -> IO ()
>   newObject o = do
>     os <- readIORef state
>     writeIORef state (o:os)
> 
>   -- used by finalizer
>   killObject :: Object -> IO ()
>   killObject o = do
>     os <- readIORef state
>     writeIORef state (filter (/= 0) os)
> 
> Some possible interleavings of the IO actions in these functions can
> result in an object not being added to or removed from the object
> list.  
[snip]
I'm not sure I really understand the problem.  The FFI standard (Release Candidate
4, the one I have printed out here) does not define IORefs, and of course 
Haskell 98 doesn't either.  Therefore, although this code is broken, this 
particular example doesn't matter  if all we are considering is code written in 
Haskell 98 + FFI.

However to me this code just looks totally wrong because of course I use GHC,
a system with preemptive scheduling, and would regard it as incompetent to use
anything other than an MVar here.  Even for Hugs I don't like this code, because
the functions newObject and killObject will fail in the future should anyone try to
port them to a system with pre-emptive concurrency, unless the maintainers are careful
to avoid running them concurrently.  So as Hugs has MVar's, I think it is bad practice
not to use them here, unless you have some very good excuse (like being desperate for
performance).  You're basically implementing an API which has the hidden condition that
the two functions must not be called simultaneously.

But in any case it's somewhat woolly just to talk in terms of existing compilers.  Supposing
we posit that there is a GlobalVariables standard.  Then indeed Alastair's example would
fall over.  So we can say that here is a problem which occurs with Haskell98 + FFI + GlobalVariables.
However I would expect a GlobalVariables standard to specify both MVars and IORefs.  
The reason is precisely because of this sort of thing.  Of course, as Alastair says, if all
you're interested in is a single-threaded implementation, MVars seems a waste of time since
either they don't block at all or they deadlock.  But they *are* very necessary if you
are intending to write code which may in the future be run on an implementation with full
concurrency.  It's precisely the same reason as why the FFI standard has a "threadsafe"
keyword; although it is these days usually unnecessary, it is nevertheless very important to have
it there because of what might happen in the future.  

So I can sum this up by saying (1) a GlobalVariables standard should specify both MVars and
IORefs.  (2) Someone writing finalizers on a Haskell98 + FFI + GlobalVariables implementation
would have absolutely no excuse for using code such as that Alastair has given, since this
is a really blinding example of where MVars are needed.  Anyone writing a finalizer should
be aware (indeed the FFI standard should say) that the finalizer may be run at any point
(after all, when else would you expect it to run) and should take precautions against it.
This is much better than expecting them to rely on arcana such as the details of Hugs
scheduling strategy.

In any case I disagree with Alastair when he says bugs of this sort are as bad as bugs caused
by calling Haskell from a finalizer in Hugs.  At least in this case Haskell's RTS is still
healthy.  Assuming the crash comes reasonably often, you can narrow it down by putting in
loads of print statements, until eventually you get to the stage of asking "Who put what into
the State variable".  Not very pleasant, but you will get there.  However if I understand it
correctly, calling Haskell from a finalizer under the FFI may lead to nothing but a core-dump.
Furthermore print statements will not help you because the finalizer might have been called 
anywhere; at the best you will only find out where the GC happened.  Even if you guess that the
reason Haskell's RTS blew up at just that point was because of a rogue finalizer during which
an external Haskell function happened to be called, you have no way in Haskell of finding
out what that finalizer might be, even less of finding out what external Haskell function the
outside world was trying to invoke.