FFI, safe vs unsafe

[Wolfgang Thaller]

John Meacham wrote:
> This doesn't have to do with bound threads, [...]

I brought it up because the implementation you are proposing  
fullfills the most important feature provided by bound threads,  
namely to be able to access the thread local state of the "main" OS  
thread (the one that runs C main ()), only for nonconcurrent calls,  
but not concurrent calls. This gives people a reason to specify some  
calls as nonconcurrent, even when they are actually expected to  
block, and it is desirable for other threads to continue running.
This creates an implementation-specific link between the concurrent/ 
nonconcurrent question and support for OS-thread-local-state. I would  
probably end up writing different code for different Haskell  
implemlementations in this situation.

Note that some predictable way of interacting with OS threads (and OS- 
thread local state) is necessary in order to be able to use some  
libraries at all, so not using OS threads *at all* might not be a  
feasible method of implementing a general-purpose programming  
language (or at least, not a feasible implementation method for  
general purpose implementations of general purpose programming  
languages).

The whole point of having bound threads is to NOT require a 1:1  
correspondence between OS threads and Haskell threads but still be  
able to interact with libraries that use OS-thread-local-state. They  
allow implementers to use OS threads just for *some* threads (i.e.  
just where necessary), while still having full efficiency and freedom  
of implementation for the other ("non-bound") threads.

There might be simpler schemes that can support libraries requiring  
OS-thread-local-state for the most common use cases, but there is a  
danger that it will interact with the concurrent/nonconcurrent issue  
in implementation-specific ways if we don't pay attention.

> I object to the idea that concurrent calls are 'safer'. getting it  
> wrong
> either way is a bug. it should fail in the most obvious way rather  
> than
> the way that can remain hidden for a long time.

How can making a call "concurrent" rather than "nonconcurrent" ever  
be a bug?

> in any case, blocking is a pretty well defined operation on operating
> systems, it is when the kernel can context switch you away waiting  
> for a
> resource, which is the main use of concurrent calls. the ability to  
> use
> them for long calculations in C is a sort of bonus, the actual main  
> use
> is to ensure the progress guarentee,

I disagree with this. First, concurrent calls serve a real-world  
purpose for all interactive programs. GUI applications are soft  
realtime systems; if a GUI application stops processing events for  
more than 2 seconds (under regular system load), I consider it buggy.
Second, although blocking is well-defined for kernel operations, the  
documented interface of most libraries does not include any  
guarantees on whether they will block the process or not; sometimes  
the difference might be entirely irrelevant; does it make a  
difference whether a drawing function in a library writes to video  
memory or sends an X request accross the network? Saying something  
"takesawhile" doesn't muddy things; it is a strictly weaker condition  
than whether something blocks.
Calculations done by foreign calls are not a "bonus", but an  
important use case for concurrent calls. Think of a library call that  
causes a multimedia library to recompress an entire video file;  
estimated time required is between a few seconds and a day. In a  
multithreaded program, this call needs to be concurrent. It is true  
that the program will still terminate even if the call is  
nonconcurrent, but in the real world, termination will probably occur  
by the user choosing to "force quit" an application that is "not  
responding" (also known as sending SIGTERM or even SIGKILL).

Reducing the issue to the question whether a function blocks or not  
is just plain wrong.

> I'd actually prefer it if there were no default and it had to
> be specified in neutral language because I think this is one of those
> things I want FFI library writers to think about.

But as I have been saying, the decision that FFI library writers have  
to make, or rather the only decision that they *can* make, is the  
simple decision of "can I guarantee that this call will return to its  
caller (or reenter Haskell via a callback) before the rest of the  
program is negatively affected by a pause". If the function could  
block, the answer is a definite "no", otherwise the question is  
inherently fuzzy. Unfortunately, I don't see a way of avoiding this  
fuzziness.

The question "can I provide a certain guarantee or not" could be  
answered with "no" by default to flatten the learning curve a bit. My  
objection against having "no default" is not very strong, but I do  
object to specifying this "in neutral language". This situation does  
not call for neutral language; rather, it has to be made clear that  
one of the options comes with a proof obligation and the other only  
with a performance penalty.

Cheers,

Wolfgang

P.S.: I'm sticking to the concurrent/nonconcurrent terminology for  
now. What terminology is really best will depend on the outcome of  
this discussion.