[Haskell-cafe] STM, IO and event loops

[Simon Marlow]

On Unix with -threaded, all blocking I/O using Handles or sockets goes
via the I/O manager thread, which multiplexes all the blocked I/O
requests into a single select() call.  If you have 20k blocked socket
reads, there will be 20k lightweight Haskell threads, and probably 2 or
3 real OS threads.  Without -threaded the outcome is pretty much the
same, except the runtime does the select().

On Windows, unfortunately you'll get 20k OS threads for this scenario
(regardless of -threaded), because we haven't implemented the I/O
manager on Windows yet.

So, to answer the question:

>> Are there any facilities in Haskell that can poll a set of 10k
>> descriptors and retrieve input from the first available?

Yes, you have one forkIO thread per descriptor, and you do your I/O
using standard Handle I/O or Network.Socket.recvfrom/sendto.
Unfortunately it'll be expensive on Windows right now, but the best way
to fix this would be to implement the I/O manager (in GHC.Conc) rather
than doing the multiplexing explicitly in your app code.

Cheers,
	Simon

On 28 November 2005 08:57, Simon Peyton-Jones wrote:

> A good design pattern, I think, is to have a Haskell thread dedicated
> to each socket.  It can suck on the socket and dump what it finds
> into an STM channel or buffer.  Then other threads (the ones doing
> the work) can read it transactionally.
> 
> It should be find to have lots of threads, esp if most of them are
> asleep.  The only thing to watch out for is that GHC's runtime system
> will consume one *OS* thread for each *blocked* foreign call.  So if
> you have 10k threads each making a separate call to the OS to read
> from 10k sockets, and they all block, you'll use 10k OS threads, and
> that will probably fail.
> 
> Simon
> 
>> -----Original Message-----
>> From: haskell-cafe-bounces at haskell.org
> [mailto:haskell-cafe-bounces at haskell.org] On Behalf Of Joel
>> Reymont
>> Sent: 27 November 2005 11:12
>> To: Haskell Cafe
>> Subject: [Haskell-cafe] STM, IO and event loops
>> 
>> Folks,
>> 
>> I'm trying to build an event-driven system that includes multiple
>> event channels. I would have one channel for network events and
>> another one for events sent by other threads. I would like to use STM
>> and `orElse` to poll the various event channels.
>> 
>> Is there a reasonably safe way to do this with STM and IO?
>> 
>> The network side of things currently times out after a given number
>> of seconds if no input is available but can also throw exceptions.
>> I'm having a hard time figuring out how to wrap this in STM.
>> 
>> There's another alternative that I can think of...
>> 
>> My poker bots are launched in separate threads but do not talk to
>> each other right now. They just receive events from the network. I
>> would like a poker bot to tell others to stop playing and exit, for
>> example. I guess I could build poker bots with an event loop that
>> reads from a TChan but... I would then need twice as many threads
>> since each bot has a socket associated with it and one thread per bot
>> would need to read from the socket and push events into the TChan.
>> 
>> Are there any facilities in Haskell that can poll a set of 10k
>> descriptors and retrieve input from the first available?
>> 
>> I don't know if I should be concerned with launching 20k threads vs.
>> 10k threads but it seems that disassociating bots from the network
>> has some positive effects. I could compose packets by hand in the
>> interpreter and feed them to the TChan by hand to test individual
>> bots, for example. 
>> 
>> Any advice is appreciated!
>> 
>> 	Thanks, Joel
>> 
>> --
>> http://wagerlabs.com/
>> 
>> 
>> 
>> 
>> 
>> _______________________________________________
>> Haskell-Cafe mailing list
>> Haskell-Cafe at haskell.org
>> http://www.haskell.org/mailman/listinfo/haskell-cafe
> _______________________________________________
> Haskell-Cafe mailing list
> Haskell-Cafe at haskell.org
> http://www.haskell.org/mailman/listinfo/haskell-cafe