Native Threads in the RTS

[Wolfgang Thaller]

Nice design, Alastair. I've stolen lots of ideas and some text for the 
complete rewrite of the proposal. The concept of "associating" haskell 
threads to native threads proved to be a good way of explaining my 
original idea in a different way --- and then I found out that 
forkNativeThread needn't be a primitive, but can be implemented on top 
of the FFI. After that I found out what the bound/free exports 
buisiness was all about and why we might need it.

As for the questions regarding my previous proposal, I think they are 
answered in the new one. If they aren't, let me know. [It's no use 
explaining what I meant to say if I already want to say something 
different ;-) ]

Cheers,

Wolfgang
===============================

Threads Proposal, version 3

Goals
~~~~~

Since foreign libraries sometimes exploit thread local state, it is
necessary to provide some control over which thread is used to execute
foreign code.  In particular, it is important that it should be
possible for Haskell code to arrange that a sequence of calls to a
given library are performed by the same native thread and that if an
external library calls into Haskell, then any outgoing calls from
Haskell are performed by the same native thread.

This specification is intended to be implementable both by
multithreaded Haskell implementations and by single-threaded
implementations and so it does not comment on which particular OS
thread is used to execute Haskell code.

Design
~~~~~~

Haskell threads may be associated at thread creation time with either
zero or one native threads. Each Native thread is associated with zero 
or one Haskell threads (no native thread may be associated with two 
Haskell threads at a time).

An associated pair of a Haskell thread and a native thread can only 
execute either foreign code or haskell code at any one time.

The thread that main runs in, threads created using forkIO and threads 
created for running finalizers or signal handlers are not associated 
with a native thread [Actually, we might make this an implementation 
detail: These Haskell threads are not guaranteed to be associated with 
a native thread. If it makes sense for some implementation, all threads 
might be associated with native threads].

There are now two kinds of foreign exported [and foreign import 
wrapped] functions: bound and free [I'm not happy with these names, for 
the same reasons as given by Seth before].

When a "bound" foreign exported function is invoked [by foreign code], 
the implementation checks whether a Haskell thread is associated with 
the current OS thread.
If there is one, this Haskell thread is used to execute the callback.
If there is none, a new Haskell thread is created and associated with 
the native thread. This is the only situation where a Haskell thread is 
associated with a native thread. The new associated Haskell thread is 
then used to execute the callback. When the callback finishes, the 
Haskell thread is terminated, the association is dissolved, but the OS 
thread continues to run.

When a "free" foreign exported function is invoked, the implementation 
may freely choose what Haskell thread the function is executed in. It 
is not specified whether this thread is associated with a particular OS 
thread or not.

When a foreign imported function is invoked [by Haskell code], the 
foreign code is executed in the native thread associated with the 
current Haskell thread, if an association exists. If the current 
Haskell thread is not associated to a native thread, the implementation 
may freely decide which thread to run the foreign function in.
This must be done for all foreign imports, including  "unsafe".
An unsafe call must not call back to Haskell or otherwise cause a 
garbage collection. Other Haskell threads may be blocked while the 
unsafe call executes.
A safe call allows callbacks to Haskell. Other Haskell threads may be 
blocked while the unsafe call executes.
A threadsafe call additionally guarantees other Haskell threads not to 
block.

forkNativeThread :: IO () -> IO ThreadID may be implemented using the 
FFI and an OS-specific thread creation routine.

Issues
~~~~~~

Finalizers and signal handlers cannot be associated with a particular 
native thread. If they have to trigger an action in a particular native 
thread, a message has to be sent manually (via MVars and friends) to 
the Haskell thread associated with the native thread in question. This 
might be tedious, but if we want to avoid the nastiness of multiple 
Haskell threads associated with one OS thread (lots of unpredictable 
blocking), it looks like our only choice.