Handling multiple fds with GHC

[David Turner]

On 7 October 2015 at 18:16, Markus Ongyerth <ongy44 at gmail.com> wrote:

> 2015-10-07 18:30 GMT+02:00 David Turner <dct25-561bs at mythic-beasts.com>:
> > Hi,
> >
> > Why the non-threaded runtime, out of interest?
>
> Mostly because i am used to the poll/select method I mentioned and that
> one works without any threading.
> I don't really mind using the threaded runtime though, it's more habit.
>
> > Threads forked with forkIO are pretty lightweight, and although things
> look
> > like blocking calls from the Haskell point of view, as I understand it
> under
> > the hood it's all done with events of one form or another. Thus even with
> > the non-threaded runtime you will see forkIO-threads behaving as if
> they're
> > running concurrently. In particular, you have two threads blocked trying
> to
> > read from two different Handles and each will be awoken just when there's
> > data to read, and the rest of the runtime will carry on even while
> they're
> > blocked. Try it!
>
> Yeah, I know and I tried that.
> As far as I can see, that's actually why things break with GHC.Event.
> The Event system tries to register the Fd while it was registered by me
> and encounters an EEXIST from epoll.
>
>
Ah, ok, so you can either do your epolling through the Haskell runtime or
with your bare hands but you can't do both on a single FD.

> If you're dealing with FDs that you've acquired from elsewhere, the
> function
> > unix:System.Posix.IO.ByteString.fdToHandle can be used to import them and
> > then they work like normal Handles in terms of blocking operations etc.
> >
> > Whenever I've had to deal with waking up for one of a number of reasons
> (not
> > all of which are FDs) I've found the simplicity of STM is hard to beat.
> > Something like:
> >
> > atomically ((Left <$> waitForFirstThing) <|> (Right <$>
> waitForSecondThing))
>
> Looks like I should look up STM. Does this scale easily?
> I don't really need huge amounts, but I don't have any knowledge about the
> number of Fds I will have.
>

Waiting on arbitrarily many things is pretty much as simple (as long as
they all have the same type so you can put them in a list):

atomically (asum listOfWaitingThings)

In terms of code complexity that scales just fine! I'm afraid I've no real
idea what the performance characteristics of such a device would be without
trying it out in your use case. Whenever I've been doing this kind of thing
I've always found myself IO-bound rather than CPU-bound so I've never found
myself worrying too much about the efficiency of the code itself.

If you're used to doing select/poll things yourself then it may help to
think of Haskell threads blocking on Handles as basically a way to do an
epoll-based event loop on the underlying FDs but with a much nicer syntax
and less mucking around with explicit continuations. Similarly, if you're
used to dealing with task scheduling at a low level then it may help to
think of STM transactions blocking as a way to muck around with the run
queues in the scheduler but with a much nicer syntax and less mucking
around with explicit continuations.



Best wishes,

David
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/glasgow-haskell-users/attachments/20151007/35c67c0a/attachment-0001.html>