[Haskell-cafe] child thread control design patterns (was: GHC.Conc.threadStatus - documentation of ThreadDied :: ThreadStatus)

[Olaf Klinke]

On Thu, 2022-03-10 at 07:44 +0000, Oliver Charles wrote:
> On Wed, 9 Mar 2022, at 7:45 PM, Olaf Klinke wrote:
> > That is indeed a neat pattern. What are its advantages/disadvantages? 
> > The main thread in my case is the (Yesod) webserver. Perhaps Yesod does
> > exactly what you propose, internally.
> 
> Mostly that one doesn't have to think about linking threads together, that just comes by design. If you want "these things must always be running", I find it's a nice fit. Also, this pattern implements the "structured concurrency" paradigm, which I'm a big fan of (see https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-considered-harmful/)
> 
> > Your idiom would be something like this?
> > 
> > main = foldr race_ (warp port site) workers
> 
> Yes, something like that.
> 
> > I also do want to include the possibility of a child (daemon) thread to
> > crash without crashing the webserver (hence my original question),
> > which your race solution does not cater for. In fact I will have a mix
> > of perpetually running actions and actions that are fired on command of
> > the user and then exit gracefully. 
> 
> Right, if you _expect_ the daemon to terminate, then this isn't a good fit. However, I would still be striving to get some kind of structured concurrency solution. To that end, I would probably have some over-arching daemon that has a shared job queue (e.g., an STM TChan), and web requests can write to this TChan (maybe with a TMVar to fill in responses). Then, the worker daemon would pop jobs off this TChan and itself spawn new threads and manage the lifetime of them. This gives you the best of both worlds - structured concurrency in main, and ephemeral worker threads.
> 
> Hope this helps,
> Ollie

Hmm, but this over-arching daemon (nursery, as it is called in the blog
post) watching over the queue *is* my webserver. Instead of a TChan it
has routes that correspond to start and pause commands of specific
daemon threads. Inter-thread signalling is indeed handled via TVars,
that's what they're for. It would be cool to have a TVar or TChan that
is read-only for one side and write-only for the other. And thank to
Control.Concurrent.Async.link I have full control over which exceptions
get propagated from child to parent. The library designers probably
should not have made async and link two distinct functions, because
that enables the programmer to disregard those exceptions. 
Should we be porting Trio to Haskell? Or does Haskell offer enough
abstraction already? 
There are some points in the blog post that I disagree with. If any
function can fork a concurrent process, not all is lost. Because of
immutable values we pretty much know what can be touched by that
concurrent process. And because of that, often we can ignore how long
the concurrent process runs because it can not affect our linear code. 

I should have read the Async documentation more carefully. 

startThread :: MyThread -> IO ()
startThread (action,var) = withAsync action (putMVar var)

immediately cancels the thread because putMVar returns. 
Hence one must use async instead of withAsync if the main thread can
not be passed to withAsync. The pattern similar to your race would be: 

foldr (\a b -> withAsync a (const b)) mainThread daemons

which, unlike race, does not kill the mainThread if one of the daemons
crashes, but cancels the daemons if the mainThread terminates. 

Olaf