FW: Lightweight Concurrency Branch

Wed Jan 4 13:30:15 UTC 2017

Hi KC,

if blackholes only appear during thunk evaluation, could the problem you
describe below be worked around by simply imposing that the scheduler never
creates black holes? Say by leveraging GHC's new -XStrict language
extension?

--
Mathieu Boespflug
Founder at http://tweag.io.

On 4 January 2017 at 13:54, Simon Peyton Jones via ghc-devs <
ghc-devs at haskell.org> wrote:

> Reply from KC… see below.
>
>
>
> Simon
>
>
>
> *From:* KC Sivaramakrishnan [mailto:sk826 at cam.ac.uk]
> *Sent:* 04 January 2017 11:29
> *To:* Simon Peyton Jones <simonpj at microsoft.com>
> *Cc:* Daniel Bennet <daniel.bennet83 at gmail.com>; ghc-devs at haskell.org; KC
> (sk826 at hermes.cam.ac.uk) <sk826 at hermes.cam.ac.uk>
> *Subject:* Re: Lightweight Concurrency Branch
>
>
>
> Hi Simon, David,
>
>
>
> Indeed as Simon mentioned it, the work was never finished. The
> implementation is(was) at a stage where we can run some small non-trivial
> benchmarks (Section 7 of the JFP paper). The interactions with FFI, MVars,
> STM, asynchronous exceptions worked well (though we most probably do things
> a little differently now), until we encountered interaction with blackholes
> mechanism.
>
>
>
> The crux of the problem is that the blackholing mechanism interacts with
> the scheduler and if the scheduler functionality itself is written in
> Haskell, then we have the potential for a deadlock. Some of the details are
> presented in Section 6.5, but we never got around to a clean solution that
> I could get working properly. Hence the reason for not formalizing it
> completely in the paper, and I am sure there are some edge cases that I
> hadn't thought about. This is a particularly tricky issue, and the current
> design unfortunately does not lend itself to a clean solution.
>
>
>
> Although the asynchronous exception mechanism interaction was not
> formalized, it works well and passes the tests. Not much effort was put in
> to make the implementation go particularly fast. While the performance was
> comparable on average, but varied quite a bit on edge cases; some of the
> results in the paper show this.
>
>
>
> If one were to revive the project, I would suggest starting from the
> design, using the existing code as a prototype, but write code from
> scratch; pleasantly there isn't much new code in this branch. The project
> does need substantial amount of work to make it upstream with the newer RTS
> mechanisms. I am very happy to provide more details and eager to assist
> with the work, but my time commitments mean that I cannot lead this effort.
>
>
>
> Kind Regards,
>
> KC
>
>
>
> On Wed, Jan 4, 2017 at 10:38 AM, Simon Peyton Jones <simonpj at microsoft.com>
> wrote:
>
> David
>
>
>
> KC never finished work on this stuff.  I’m copying him because I’m sure
> he’d be happy to help.
>
>
>
> KC: can you summarise where you left it?
>
>
>
> I think it’s very interesting work, and has the potential to make GHC’s
> RTS much more malleable, by moving more of it into Haskell libraries
> instead of deeply-magic C code.
>
>
>
> But it’s not easy, because we are reluctant to lose performance, and
> because there are interactions with STM, weak pointers, foreign function
> calls, etc.   I think it’d require a bit of commitment to make a go of it.
>
>
>
>
>
> Simon
>
>
>
>
>
> *From:* ghc-devs [mailto:ghc-devs-bounces at haskell.org] *On Behalf Of *Daniel
> Bennet
> *Sent:* 28 December 2016 17:23
> *To:* ghc-devs at haskell.org
> *Subject:* Lightweight Concurrency Branch
>
>
>
> The lightweight concurrency branch is highly interesting and relevant to
> my interests, however, the ghc-lwc2 branch hasn't been updated in several
> years even though it's listed as an active branch at
> https://ghc.haskell.org/trac/ghc/wiki/ActiveBranches
>
>
>
> The wiki page for the work hasn't been updated in almost two years either,
> https://ghc.haskell.org/trac/ghc/wiki/LightweightConcurrency
>
>
>
> Relevant papers:
>
> Composable Scheduler Activations for Haskell (2014)
>
> https://timharris.uk/papers/2014-composable-tr.pdf
> <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftimharris.uk%2Fpapers%2F2014-composable-tr.pdf&data=02%7C01%7Csimonpj%40microsoft.com%7Cd132a48641a54f732ffc08d42f463e8b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C1%7C636185426136080161&sdata=YG%2B0xfi%2Bd74Vi9GBI8iTrr4GEMzjS2WwrI182%2BcEgqM%3D&reserved=0>
>
>
>
>
> Composable Scheduler Activations for Haskell (2016)
>
> http://kcsrk.info/papers/schedact_jfp16.pdf
> <https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fkcsrk.info%2Fpapers%2Fschedact_jfp16.pdf&data=02%7C01%7Csimonpj%40microsoft.com%7Cd132a48641a54f732ffc08d42f463e8b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C1%7C636185426136080161&sdata=mH79Y3U%2BdDvcw2PA8lGcFaPMJw6DgCIXbu%2F3WGvyGaU%3D&reserved=0>
>
>
>
> What remains for integrating this branch into GHC?
>
>
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20170104/620723fa/attachment.html>