[Haskell-cafe] External system connections

[Richard Wallace]

On Sun, Jul 10, 2011 at 4:38 PM, Brandon Allbery <allbery.b at gmail.com> wrote:
> One of the reasons to send the request back to the dispatcher instead
> of doing it inline is so that the dispatcher can note that a renewal
> request is already in flight (which it needs to know anyway, so it can
> block other requests) and wake all threads waiting on it when it's
> done, instead of having multiple renewals in flight.
>

Ok, I can see that.  Though I was thinking that the worker threads
would send the request back to the dispatcher by way of the same Chan
the dispatcher reads requests from.  Obviously I was thinking the
dispatcher would use the original token to filter requests in the
Chan.  If I understand what you are talking about, the dispatcher
would do the token renewal in a separate thread, continuing to process
it's incoming Chan while that is going on, accumulating incoming
requests somewhere, perhaps in another Chan.  Then, when the token
renewal is complete, the dispatcher stops forwarding incoming requests
to the secondary Chan, processes the accumulated requests using the
new token, and, when done with those, the switches back to processing
the incoming Chan.

Is that something like what you had in mind, or did I make it more
complicated than necessary?

> The point of a pool is so (a) you can throttle it in special cases,
> such as when you need to renew the token, and (b) so you don't find
> yourself juggling a couple thousand threads if things get unexpectedly
> busy (or buggy).  You can limit the pool to something sensible
> (probably something like 4 during development and debugging so things
> can't get too out of hand) and increased later; plus, the pool manager
> will provide the primitives to deal with managing shared resources
> (such as your token) within the pool.
>

Hmm.  I'm still not entirely seeing it and I think the problem is just
my lack of knowledge of Haskell concurrency.

If the pool is of threads, how do you start the threads?  How do you
submit work to the threads?  The only way I know of in Haskell of
creating threads to do work is forkIO.  That takes a function and runs
to completion.  Would a worker thread just be one that loops forever
and checks a MVar for something to do?  So the pool would really
consist of (MVar, ThreadId) pairs.  When we speak of getting a thread
out of a pool and giving it work to do, are we really talking about
sticking a value in an MVar that the thread is blocking on, waiting
for data to be available to do something with, so that it can go and
do some work?  And when we are done with the thread we return it to
the pool?

Rich

P.S. Thanks for talking me through this!  I'm learning a ton about
concurrency in Haskell.

> --
> brandon s allbery                                      allbery.b at gmail.com
> wandering unix systems administrator (available)     (412) 475-9364 vm/sms
>