FFI, safe vs unsafe

[John Meacham]

On Wed, Apr 12, 2006 at 07:35:22PM -0400, Wolfgang Thaller wrote:
> John Meacham wrote:
> >This doesn't have to do with bound threads, [...]
> 
> I brought it up because the implementation you are proposing  
> fullfills the most important feature provided by bound threads,  
> namely to be able to access the thread local state of the "main" OS  
> thread (the one that runs C main ()), only for nonconcurrent calls,  
> but not concurrent calls. This gives people a reason to specify some  
> calls as nonconcurrent, even when they are actually expected to  
> block, and it is desirable for other threads to continue running.
> This creates an implementation-specific link between the concurrent/ 
> nonconcurrent question and support for OS-thread-local-state. I would  
> probably end up writing different code for different Haskell  
> implemlementations in this situation.

Oh, I made that proposal a while ago as a first draft, bound threads
should be possible whether calls are concurrent or not, I am not
positive I like the ghc semantics, but bound threads themselves pose not
much more overhead than supporting concurrent in the first place (which
is a fairly substantial overhead to begin with). but that doesn't matter
to me if there isn't a performance impact in the case where they arn't
used. 

However, in order to achieve that we would have to annotate the foreign
functions with whether they use thread local state. it would pretty much
be vital for implementing them efficiently on a non OS-threaded
implemenation of the language. you need to perform a
stack-pass-the-baton dance between threads to pass the haskell stack to
the right OS thread which is a substantial overhead you can't pay just
in case it might be running in a 'forkOS' created thread. Checking
thread local state for _every_ foregin call is definitly not an option
either. (but for specificially annotated ones it is fine.) ghc doesn't
have this issue because it can have multiple haskell threads running at
once on different OS threads, so it just needs to create one that
doesn't jump between threads and let foreign calls proceed naturally.
non-os threaded implementations have the opposite problem, they need to
support a haskell thread that _can_ (and does) jump between OS threads.
one pays the cost at thread creation time, the other pays the cost at
foreign call time. the only way to reconcile these would be to annotate
both. (which is perfectly fine by me if bound threads are needed, which I
presume they are)

Oddly enough, depending on the implementation it might actually be
easier to just make every 'threadlocal' function fully concurrent. you
have already paid the cost of dealing with OS threads.

> Calculations done by foreign calls are not a "bonus", but an  
> important use case for concurrent calls. Think of a library call that  
> causes a multimedia library to recompress an entire video file;  
> estimated time required is between a few seconds and a day. In a  
> multithreaded program, this call needs to be concurrent. It is true  
> that the program will still terminate even if the call is  
> nonconcurrent, but in the real world, termination will probably occur  
> by the user choosing to "force quit" an application that is "not  
> responding" (also known as sending SIGTERM or even SIGKILL).

they are a bonus in that you can't run concurrent computing haskell
threads at the same time. you get "free" concurrent threads in other
languages that you would not get if the libraries just happened to be
implemented in haskell. However, if the libraries were implemented in
haskell, you would still get concurrency on OS blocking events because
the progress guarentee says so.

> The question "can I provide a certain guarantee or not" could be  
> answered with "no" by default to flatten the learning curve a bit. My  
> objection against having "no default" is not very strong, but I do  
> object to specifying this "in neutral language". This situation does  
> not call for neutral language; rather, it has to be made clear that  
> one of the options comes with a proof obligation and the other only  
> with a performance penalty.

you seem to be contradicting yourself, above you say a performance
penalty is vitally important in the GUI case if a call takes too long,
but here you call it 'just a performance penalty'. The overhead of
concurrent calls is quite substantial. Who is to say whether a app that
muddles along is better or worse than one that is generally snappy but
has an occasional delay.

Though, I am a fan of neutral language in general. you can't crash the
system like you can with unsafePerformIO, FFI calls that take a while
and arn't already wrapped by the standard libraries are relatively rare.
no need for strong language.


        John


-- 
John Meacham - ⑆repetae.net⑆john⑈