Safe FFI and Blocking IO

[Andrew Martin]

Sorry. I just found the answer to this in the manual:

"When you call a foreign imported function that is annotated as safe (the
default), and the program was linked using -threaded, then the call will
run concurrently with other running Haskell threads. If the program was
linked without -threaded, then the other Haskell threads will be blocked
until the call returns."

"This means that if you need to make a foreign call to a function that
takes a long time or blocks indefinitely, then you should mark it safe and
use -threaded. Some library functions make such calls internally; their
documentation should indicate when this is the case."



On Tue, Dec 4, 2018 at 8:23 AM Andrew Martin <andrew.thaddeus at gmail.com>
wrote:

> According to the FFI chapter [1] in the GHC manual, the safe FFI is useful
> when you need to call a C function that can call back into haskell code. I
> had always assumed that the scheduler could somehow interrupt safe FFI
> calls, but the manual does not indicate this, and in some recent testing I
> did in the posix library [2], I found that scheduling interrupts definitely
> do not happen. With the non-threaded runtime, the following test always
> hangs:
>
>     testSocketsD :: IO ()
>     testSocketsD = do
>       (a,b) <- demand =<< S.socketPair P.unix P.datagram P.defaultProtocol
>       _ <- forkIO $ do
>         bytesSent <- demand =<< S.sendByteArray b sample 0 5 mempty
>         when (bytesSent /= 5) (fail "testSocketsD: bytesSent was wrong")
>       actual <- demand =<< S.receiveByteArray a 5 mempty
>       actual @=? sample
>
>     sample :: ByteArray
>     sample = E.fromList [1,2,3,4,5]
>
>     demand :: Either Errno a -> IO a
>     demand = either (\e -> ioError (errnoToIOError "test" e Nothing
> Nothing)) pure
>
> In the above example, sendByteArray and receiveByteArray are safe FFI
> wrappers around send and recv. It is necessary to use threadWaitRead and
> threadWaitWrite before these calls to predictably get the correct behavior.
>
> This brings to my question. In issue #34 on the github library for the
> unix package [3], there is a discussion about whether to use the safe or
> unsafe FFI for various POSIX system calls. On the issue there is strong
> consensus that the safe FFI calls lead to better performance.
>
> Simon Marlow writes [4] that "Unsafe foreign imports which can block for
> unpredictable amounts of time cause performance problems that only emerge
> when scaling to multiple cores, because they delay the GC sync. This is a
> really annoying problem if it happens to you, because it's almost
> impossible to diagnose, and if it happens due to an unsafe call in a
> library then it's also really hard to fix."
>
> And Gregory Collins adds that "If the call would ever block (and that
> includes most filesystem functions) that means you want 'safe'."
>
> There's something I'm definitely missing. My experience is that safe FFI
> calls do not help with blocking IO (again, I've been using the non-threaded
> runtime, but I doubt this makes a difference), that they only help with C
> functions that call back into haskell. However, a lot of other people seem
> to have a difference experience.
>
> [1]
> https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/ffi-chap.html#foreign-function-interface-ffi
> [2] https://github.com/andrewthad/posix
> [3] https://github.com/haskell/unix/issues/34
> [4] https://github.com/haskell/unix/issues/34#issuecomment-68683424
>
> --
> -Andrew Thaddeus Martin
>


-- 
-Andrew Thaddeus Martin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/libraries/attachments/20181204/b496c3fb/attachment.html>