<div dir="ltr">What's your heuristic for deciding between safe and interruptible? I find that every time something takes long enough to warrant using the safe FFI, I also want to be able to kill it from a separate thread.</div><br><div class="gmail_quote"><div dir="ltr">On Tue, Dec 4, 2018 at 9:58 AM Carter Schonwald <<a href="mailto:carter.schonwald@gmail.com">carter.schonwald@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div dir="ltr">yup! (this is also kinda related to how the IO manager only runs on -threaded built applications)<div><br></div><div>I actually do the following pattern in some libraries i've written: bind both unsafe and safe versions of a functions, and when work input is below some size that i think will be less than ~ 1-10 microseconds, i do an unsafe call, otherwise i do a safe call! (unsafe calls block the GC, which is bad in say a server app, as you can well guess)</div><div><br></div><div>the most recent and tiny example of this is a tiny sha3 implementation (still need to tweak it, i think i left another 4-6x performance on the table <a href="https://hackage.haskell.org/package/SecureHash-SHA3" target="_blank">https://hackage.haskell.org/package/SecureHash-SHA3</a>)</div></div></div><br><div class="gmail_quote"><div dir="ltr">On Tue, Dec 4, 2018 at 8:26 AM Andrew Martin <<a href="mailto:andrew.thaddeus@gmail.com" target="_blank">andrew.thaddeus@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Sorry. I just found the answer to this in the manual:<div><br>"When you call a foreign imported function that is annotated as safe (the default), and the program was linked using -threaded, then the call will run concurrently with other running Haskell threads. If the program was linked without -threaded, then the other Haskell threads will be blocked until the call returns."<br><br>"This means that if you need to make a foreign call to a function that takes a long time or blocks indefinitely, then you should mark it safe and use -threaded. Some library functions make such calls internally; their documentation should indicate when this is the case."</div><div></div><div><br></div><div><br></div></div><br><div class="gmail_quote"><div dir="ltr">On Tue, Dec 4, 2018 at 8:23 AM Andrew Martin <<a href="mailto:andrew.thaddeus@gmail.com" target="_blank">andrew.thaddeus@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr">According to the FFI chapter [1] in the GHC manual, the safe FFI is useful when you need to call a C function that can call back into haskell code. I had always assumed that the scheduler could somehow interrupt safe FFI calls, but the manual does not indicate this, and in some recent testing I did in the posix library [2], I found that scheduling interrupts definitely do not happen. With the non-threaded runtime, the following test always hangs:</div><div dir="ltr"><br></div><div dir="ltr"><div dir="ltr"> testSocketsD :: IO ()</div><div dir="ltr"> testSocketsD = do</div><div dir="ltr"> (a,b) <- demand =<< S.socketPair P.unix P.datagram P.defaultProtocol</div><div dir="ltr"> _ <- forkIO $ do</div><div dir="ltr"> bytesSent <- demand =<< S.sendByteArray b sample 0 5 mempty</div><div dir="ltr"> when (bytesSent /= 5) (fail "testSocketsD: bytesSent was wrong")</div><div dir="ltr"> actual <- demand =<< S.receiveByteArray a 5 mempty</div><div dir="ltr"> actual @=? sample</div><div><br></div></div><div dir="ltr"><div dir="ltr"> sample :: ByteArray</div><div dir="ltr"> sample = E.fromList [1,2,3,4,5]</div><div dir="ltr"><br></div><div dir="ltr"> demand :: Either Errno a -> IO a</div><div dir="ltr"> demand = either (\e -> ioError (errnoToIOError "test" e Nothing Nothing)) pure</div><div><br></div><div>In the above example, sendByteArray and receiveByteArray are safe FFI wrappers around send and recv. It is necessary to use threadWaitRead and threadWaitWrite before these calls to predictably get the correct behavior.</div><div><br></div><div>This brings to my question. In issue #34 on the github library for the unix package [3], there is a discussion about whether to use the safe or unsafe FFI for various POSIX system calls. On the issue there is strong consensus that the safe FFI calls lead to better performance.</div><div><br></div><div>Simon Marlow writes [4] that "Unsafe foreign imports which can block for unpredictable amounts of time cause performance problems that only emerge when scaling to multiple cores, because they delay the GC sync. This is a really annoying problem if it happens to you, because it's almost impossible to diagnose, and if it happens due to an unsafe call in a library then it's also really hard to fix."</div><br>And Gregory Collins adds that "If the call would ever block (and that includes most filesystem functions) that means you want 'safe'."</div><div dir="ltr"><br></div><div dir="ltr">There's something I'm definitely missing. My experience is that safe FFI calls do not help with blocking IO (again, I've been using the non-threaded runtime, but I doubt this makes a difference), that they only help with C functions that call back into haskell. However, a lot of other people seem to have a difference experience.<br><div><br></div><div>[1] <a href="https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/ffi-chap.html#foreign-function-interface-ffi" target="_blank">https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/ffi-chap.html#foreign-function-interface-ffi</a></div><div>[2] <a href="https://github.com/andrewthad/posix" target="_blank">https://github.com/andrewthad/posix</a></div><div>[3] <a href="https://github.com/haskell/unix/issues/34" target="_blank">https://github.com/haskell/unix/issues/34</a></div><div>[4] <a href="https://github.com/haskell/unix/issues/34#issuecomment-68683424" target="_blank">https://github.com/haskell/unix/issues/34#issuecomment-68683424</a></div><div><br></div><div>-- <br><div dir="ltr" class="m_7073735783816910947m_-1031526463013216850m_-689408865270557903gmail_signature">-Andrew Thaddeus Martin</div></div></div></div></div></div></div></div></div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="m_7073735783816910947m_-1031526463013216850gmail_signature" data-smartmail="gmail_signature">-Andrew Thaddeus Martin</div>
_______________________________________________<br>
Libraries mailing list<br>
<a href="mailto:Libraries@haskell.org" target="_blank">Libraries@haskell.org</a><br>
<a href="http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries" rel="noreferrer" target="_blank">http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries</a><br>
</blockquote></div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature">-Andrew Thaddeus Martin</div>