[Haskell-cafe] FFI and callbacks

[Duncan Coutts]

On Wed, 2005-07-20 at 16:10 +0100, Simon Marlow wrote:
> On 20 July 2005 14:35, John Goerzen wrote:
> 
> > I'm looking at packaging an event-driven console widget set (CDK) for
> > Haskell using FFI.  I know that other event-driven widget sets have
> > Haskell bindings, but I'm not quite sure how to make everything play
> > nice with forkIO.
> > 
> > These systems generally have some sort of an opaque main loop,
> > implemented in C.  This loop would usually never return, or perhaps
> > only return once the UI is destroyed.
> 
> Is the library thread-safe or not?  I mean OS-thread safe.  If it is,
> then you're home and dry: just compile your program with GHC's -threaded
> option, and any foreign calls that need to run concurrently with Haskell
> threads must be declared "safe" (which is the default in fact, so
> instead I should really advise you to mark all calls that don't need to
> run concurrently as "unsafe", because that's good for performance).
> 
> If the library isn't thread-safe, then you're in the same boat as Gtk
> and wxHaskell (I believe) where there are known problems with having a
> multithreaded Haskell GUI app.

Yep.

> You can only have one OS thread (and hence only one Haskell thread)
> doing GUI operations, and that is the thread in the main loop.

To be more precise, they can only be acessed from one OS thread. You can
use multiple Haskell threads so long as they only make GUI calls from
within the main OS thread.

The problem then as John noted is that the main loop of these toolkits
block and so the other Haskell threads would not get a chance to
schedule. So the challenge is to give the Haskell threads a chance to
schedule.

Most toolkits with a main loop system allow you to setup timers. In the
Gtk2Hs bindings we can use this trick:

-- 50ms timeout, so GHC will get a chance to scheule about 20 times a second
-- which gives reasonable latency without the polling generating too much
-- cpu load.
timeoutAddFull (yield >> return True) priorityDefaultIdle 50

This causes the Gtk main loop to yeild to the ghc rts and give it a
chance to run any runnable threads for a while. It is unfortunately a
polling solution which has it's obvious disadvantages. However in
practice it seems to generate negligable load when idle. The GHC rts
yield function seems to be quite good about not doing much work when
there is no work to be done!

With this trick you can then use forkIO as much as you like and all
Haskell threads can make GUI calls. We have a nice multi-threaded IRC
client example, where one thread listens to incomming network traffic
and updates the GUI from there (the main thread is in the Gtk main loop
of course).

However, this is all assuming you're using the single threaded ghc rts.
If you link using -threaded then this all goes wrong. This is because
with -threaded we can no longer guarantee the all the Haskell threads
that want to make GUI calls will do so from the main OS thread
(specifically the one that originally entered the Gtk main loop).

Unfortnately the current bound threads system does not help. The current
bound threads system allows you to fork a new Haskell thread bound to a
*new* OS thread. This is just what you need for OpenGL. However for UI
toolkits like Gtk, wxWidgets, CDK etc we need to fork a new Haskell
thread bound to an *existing* OS thread.

If we could do that then the existing mechanism would continue to work. 

To eliminate the polling and do something more satisfactory would
probably require some support from the RTS to find out the event sources
(file handles) and timers that would cause the RTS to wake up.

> You have to somehow set up a communication between your other Haskell
> threads and the thread running the main loop - perhaps you can send
> requests of type (IO ()) down a channel to the main loop thread which
> wakes up occasionally to run them, for example.

I think that sending every GUI request through a pipe and across a
thread boundary would be a great deal of overhead. Many GUI calls are
quite fine grained.

Duncan