Handling multiple fds with GHC

[David Turner]

Hi,

Why the non-threaded runtime, out of interest?

Threads forked with forkIO are pretty lightweight, and although things look
like blocking calls from the Haskell point of view, as I understand it
under the hood it's all done with events of one form or another. Thus even
with the non-threaded runtime you will see forkIO-threads behaving as if
they're running concurrently. In particular, you have two threads blocked
trying to read from two different Handles and each will be awoken just when
there's data to read, and the rest of the runtime will carry on even while
they're blocked. Try it!

If you're dealing with FDs that you've acquired from elsewhere, the
function unix:System.Posix.IO.ByteString.fdToHandle can be used to import
them and then they work like normal Handles in terms of blocking operations
etc.

Whenever I've had to deal with waking up for one of a number of reasons
(not all of which are FDs) I've found the simplicity of STM is hard to
beat. Something like:

atomically ((Left <$> waitForFirstThing) <|> (Right <$> waitForSecondThing))

where waitForFirstThing and waitForSecondThing are blocked waiting for
something interesting to occur in a TVar that they're watching. It's so
simple that I reckon it's worth doing it like that and only trying
something more complicated if it turns out from experimentation that this
has too much overhead for you - "make it right" precedes "make it fast".

Hope that helps,

David




On 7 October 2015 at 08:49, Markus Ongyerth <ongy44 at gmail.com> wrote:

> Hi,
>
> the last few days, I tried to get an IO-Event system running with GHC i.e.
> trigger an IO action when there is data to read from a fd.
> I looked at a few different implementations, but all of them have some
> downside.
>
>  * using select package
>    - This uses the select syscall. select is rather limited (fd cannot be
> >1024)
>
>  * using GHC.Event
>    - GHC.Event is broken in 7.10.1 (unless unsafeCoerce and a hacky trick
> are used)
>    - GHC.Event is GHC internal according to hackage
>    - Both Network libraries I looked at (networking (Network.Socket) and
> socket (System.Socket)) crash the application with GHC.Event
>     - with 7.8+ I didn't see a way to create your own EventManager, so it
> only works with -threaded
>
>  * using forkIO and threadWaitRead for each fd in a loop
>     - needs some kind of custom control structure around it
>     - uses a separate thread for each fd
>     - might become pretty awkward to handle multiple events
>
>  * using poll package
>    - blocks in a safe foreign call
>    - needs some kind of wrapper
>
>
> From the above list, GHC.Event isn't usable (for me) right now. It would
> require some work for my usecase.
> The select option is usable, but suffers from the same problems as poll +
> the limitation mentioned, so it is strictly worse.
>
> This leaves me with two options: poll and forkIO + blocking.
>
> Those are based on two completely different approaches to event handling..
>
> poll can be used in a rather classic event handling system with a main
> loop that blocks until an event occurs (or a timeout triggers) and handles
> the event in the loop.
> forkIO + blocking is closer to registering an action later that should be
> triggered by an event.
>
> My main questions right now are:
> 1. How bad is it for the (non-threaded) runtime to be blocking in a
> foreign call most of the time?
> 2. How significant will the overhead be for the forkIO version?
> 3. Is there a *good* way to use something like threadWaitRead that allows
> to wake up on other events as well?
> 4. Is there a better way to handle multiple fds that may get readable data
> at any time, in Haskell/with GHC right now?
>
> Thanks in advance,
> Ongy
>
> _______________________________________________
> Glasgow-haskell-users mailing list
> Glasgow-haskell-users at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/glasgow-haskell-users/attachments/20151007/75e7f88a/attachment.html>