Handling multiple fds with GHC

[Markus Ongyerth]

2015-10-07 21:17 GMT+02:00 David Turner <dct25-561bs at mythic-beasts.com>:
>
>
> On 7 October 2015 at 18:16, Markus Ongyerth <ongy44 at gmail.com> wrote:
>>
>> 2015-10-07 18:30 GMT+02:00 David Turner <dct25-561bs at mythic-beasts.com>:
>> > Hi,
>> >
>> > Why the non-threaded runtime, out of interest?
>>
>> Mostly because i am used to the poll/select method I mentioned and that
>> one works without any threading.
>> I don't really mind using the threaded runtime though, it's more habit.
>>
>> > Threads forked with forkIO are pretty lightweight, and although things
>> > look
>> > like blocking calls from the Haskell point of view, as I understand it
>> > under
>> > the hood it's all done with events of one form or another. Thus even
>> > with
>> > the non-threaded runtime you will see forkIO-threads behaving as if
>> > they're
>> > running concurrently. In particular, you have two threads blocked trying
>> > to
>> > read from two different Handles and each will be awoken just when
>> > there's
>> > data to read, and the rest of the runtime will carry on even while
>> > they're
>> > blocked. Try it!
>>
>> Yeah, I know and I tried that.
>> As far as I can see, that's actually why things break with GHC.Event.
>> The Event system tries to register the Fd while it was registered by me
>> and encounters an EEXIST from epoll.
>>
>
> Ah, ok, so you can either do your epolling through the Haskell runtime or
> with your bare hands but you can't do both on a single FD.

Ah, I didn't to it with bare hands, I did it with GHC.Event registerFd.
Running my own epoll might work (according to the epoll man page),
but I really don't want to do that.

>> > If you're dealing with FDs that you've acquired from elsewhere, the
>> > function
>> > unix:System.Posix.IO.ByteString.fdToHandle can be used to import them
>> > and
>> > then they work like normal Handles in terms of blocking operations etc.
>> >
>> > Whenever I've had to deal with waking up for one of a number of reasons
>> > (not
>> > all of which are FDs) I've found the simplicity of STM is hard to beat.
>> > Something like:
>> >
>> > atomically ((Left <$> waitForFirstThing) <|> (Right <$>
>> > waitForSecondThing))
>>
>> Looks like I should look up STM. Does this scale easily?
>> I don't really need huge amounts, but I don't have any knowledge about the
>> number of Fds I will have.
>
>
> Waiting on arbitrarily many things is pretty much as simple (as long as they
> all have the same type so you can put them in a list):
>
> atomically (asum listOfWaitingThings)

Oh, I didn't see asum, but "came up" with the same implementation.

> In terms of code complexity that scales just fine! I'm afraid I've no real
> idea what the performance characteristics of such a device would be without
> trying it out in your use case. Whenever I've been doing this kind of thing
> I've always found myself IO-bound rather than CPU-bound so I've never found
> myself worrying too much about the efficiency of the code itself.
>
> If you're used to doing select/poll things yourself then it may help to
> think of Haskell threads blocking on Handles as basically a way to do an
> epoll-based event loop on the underlying FDs but with a much nicer syntax
> and less mucking around with explicit continuations. Similarly, if you're
> used to dealing with task scheduling at a low level then it may help to
> think of STM transactions blocking as a way to muck around with the run
> queues in the scheduler but with a much nicer syntax and less mucking around
> with explicit continuations.
>

For my current project the speed does not really matter, but I tend to do some
research anyway, since I might get to a point where I need it.

The one thing I am not sure about right now, is how to use threadWaitReadSTM.
Can I reuse the STM? I have two Fds I can test it with right now, and
one of them
works, the other one doesn't seem to work for me. I looked into the
source and to
me it looks like the STM should not be reused, since the content of
the TVar used
internally will be set to True.

Thanks for the help,

ongy