[Haskell-cafe] External system connections

[Richard Wallace]

Alright, I'll have to think on this some more but I think we're
speaking the same language now - and what's more I even understand it!

Thanks again for all your help,
Rich

On Sun, Jul 10, 2011 at 6:46 PM, Brandon Allbery <allbery.b at gmail.com> wrote:
> On Sun, Jul 10, 2011 at 20:19, Richard Wallace
> <rwallace at thewallacepack.net> wrote:
>> On Sun, Jul 10, 2011 at 4:38 PM, Brandon Allbery <allbery.b at gmail.com> wrote:
>> Ok, I can see that.  Though I was thinking that the worker threads
>> would send the request back to the dispatcher by way of the same Chan
>> the dispatcher reads requests from.  Obviously I was thinking the
>> dispatcher would use the original token to filter requests in the
>> Chan.  If I understand what you are talking about, the dispatcher
>> would do the token renewal in a separate thread, continuing to process
>
> The same Chan is used to send a request; the thread processing token
> renewal might or might not be otherwise a normal worker thread, but
> it's separated out as a Maybe ThreadId instead of being in a pool,
> because (a) there can only be zero or one of them, and (b) if it's not
> Nothing then the dispatcher thread accepts only token renewals.  (This
> actually requires either multiple Chans or something more complex than
> a normal Chan, since you can't filter a Chan based on the type of
> message.)  You also need some way to block the sender, which suggests
> that a message written down a Chan must include an MVar which will be
> signaled when the operation is complete.  This suggests to me
> something along the lines of
>
>> data WorkRequest = SOAPData ... (MVar Bool)
>>                  | TokenRequest Token (MVar Token)
>> --               | ReadyForWork (MVar WorkRequest)
>
> where the requestor allocates an MVar, writes it as part of the
> WorkRequest, and then does a takeMVar to wait for the response.  The
> dispatcher reads WorkRequests, dispatches any it can to available
> workers, and queues the rest internally; if it's a TokenRequest then
> it's queued separately and all SOAPData requests get queued regardless
> of whether there are free workers.  When the single token processor
> returns, all entries in the TokenRequest queue get awakened (putMVar
> threadMVar newToken) and normal processing of the standard request
> queue resumes.
>
> Or you can see if the pool hackage handles the ugly details here
> automatically; I haven't looked.
>
>> If the pool is of threads, how do you start the threads?  How do you
>> submit work to the threads?  The only way I know of in Haskell of
>> creating threads to do work is forkIO.  That takes a function and runs
>> to completion.  Would a worker thread just be one that loops forever
>
> Yes; the dispatcher keeps a list of workers, which are forkIO-d
> threads that are waiting on an MVar or Chan for work to do.  When they
> receive something, they go off and do it, write the result into
> another MVar or Chan which was specified in the request, and go back
> to waiting on the initial MVar/Chan for something to do.  If the list
> is shorter than the maximum, more workers are forkIO-d to fill it as
> needed; if longer, idle workers are sent "shut down" requests.  (The
> latter is "polite" handling of program shutdown, and also allows for
> the pool size to be modified dynamically if needed.)  I think doing
> this right also requires that a worker that's ready for more work
> explicitly check in, so the dispatcher knows it's available; that
> could be handled by an additional WorkRequest type (see commented-out
> line above, where a worker that's ready to handle another request
> passes its input MVar to the dispatcher)... but there may be better
> ways; I have some grasp of concurrency, but my Haskell library fu is
> still somewhat weak.  Hopefully someone else will jump in if
> appropriate.
>
> (You can see how quickly this becomes complex, though; if the canned
> solution does what you need, you might want to avoid reinventing this
> particular wheel unless you're doing it for educational purposes.)
>
> --
> brandon s allbery                                      allbery.b at gmail.com
> wandering unix systems administrator (available)     (412) 475-9364 vm/sms
>