FW: Lightweight Concurrency Branch

[Simon Peyton Jones]

Reply from KC… see below.

Simon

From: KC Sivaramakrishnan [mailto:sk826 at cam.ac.uk]
Sent: 04 January 2017 11:29
To: Simon Peyton Jones <simonpj at microsoft.com>
Cc: Daniel Bennet <daniel.bennet83 at gmail.com>; ghc-devs at haskell.org; KC (sk826 at hermes.cam.ac.uk) <sk826 at hermes.cam.ac.uk>
Subject: Re: Lightweight Concurrency Branch

Hi Simon, David,

Indeed as Simon mentioned it, the work was never finished. The implementation is(was) at a stage where we can run some small non-trivial benchmarks (Section 7 of the JFP paper). The interactions with FFI, MVars, STM, asynchronous exceptions worked well (though we most probably do things a little differently now), until we encountered interaction with blackholes mechanism.

The crux of the problem is that the blackholing mechanism interacts with the scheduler and if the scheduler functionality itself is written in Haskell, then we have the potential for a deadlock. Some of the details are presented in Section 6.5, but we never got around to a clean solution that I could get working properly. Hence the reason for not formalizing it completely in the paper, and I am sure there are some edge cases that I hadn't thought about. This is a particularly tricky issue, and the current design unfortunately does not lend itself to a clean solution.

Although the asynchronous exception mechanism interaction was not formalized, it works well and passes the tests. Not much effort was put in to make the implementation go particularly fast. While the performance was comparable on average, but varied quite a bit on edge cases; some of the results in the paper show this.

If one were to revive the project, I would suggest starting from the design, using the existing code as a prototype, but write code from scratch; pleasantly there isn't much new code in this branch. The project does need substantial amount of work to make it upstream with the newer RTS mechanisms. I am very happy to provide more details and eager to assist with the work, but my time commitments mean that I cannot lead this effort.

Kind Regards,
KC

On Wed, Jan 4, 2017 at 10:38 AM, Simon Peyton Jones <simonpj at microsoft.com<mailto:simonpj at microsoft.com>> wrote:
David

KC never finished work on this stuff.  I’m copying him because I’m sure he’d be happy to help.

KC: can you summarise where you left it?

I think it’s very interesting work, and has the potential to make GHC’s RTS much more malleable, by moving more of it into Haskell libraries instead of deeply-magic C code.

But it’s not easy, because we are reluctant to lose performance, and because there are interactions with STM, weak pointers, foreign function calls, etc.   I think it’d require a bit of commitment to make a go of it.


Simon


From: ghc-devs [mailto:ghc-devs-bounces at haskell.org<mailto:ghc-devs-bounces at haskell.org>] On Behalf Of Daniel Bennet
Sent: 28 December 2016 17:23
To: ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>
Subject: Lightweight Concurrency Branch

The lightweight concurrency branch is highly interesting and relevant to my interests, however, the ghc-lwc2 branch hasn't been updated in several years even though it's listed as an active branch at https://ghc.haskell.org/trac/ghc/wiki/ActiveBranches

The wiki page for the work hasn't been updated in almost two years either, https://ghc.haskell.org/trac/ghc/wiki/LightweightConcurrency

Relevant papers:
Composable Scheduler Activations for Haskell (2014)
https://timharris.uk/papers/2014-composable-tr.pdf<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftimharris.uk%2Fpapers%2F2014-composable-tr.pdf&data=02%7C01%7Csimonpj%40microsoft.com%7Cd132a48641a54f732ffc08d42f463e8b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C1%7C636185426136080161&sdata=YG%2B0xfi%2Bd74Vi9GBI8iTrr4GEMzjS2WwrI182%2BcEgqM%3D&reserved=0>

Composable Scheduler Activations for Haskell (2016)
http://kcsrk.info/papers/schedact_jfp16.pdf<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fkcsrk.info%2Fpapers%2Fschedact_jfp16.pdf&data=02%7C01%7Csimonpj%40microsoft.com%7Cd132a48641a54f732ffc08d42f463e8b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C1%7C636185426136080161&sdata=mH79Y3U%2BdDvcw2PA8lGcFaPMJw6DgCIXbu%2F3WGvyGaU%3D&reserved=0>

What remains for integrating this branch into GHC?

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20170104/6d533e02/attachment-0001.html>