Changes in GHC API modularity with the "Encode shape information in PMOracle" MR

[Sebastian Graf]

Hmm. The issue of build parallelism is kind-of orthogonal to .hs-boot
files/circular imports.

The impact of .hs-boot files on build parallelism depends on their
announced imports; at least in my semantic model of them (which might be
incorrect) they just break cycles.
The degree of parallelism is visible as a property of this DAG where cycles
are broken through insertion of .hs-boot nodes.

The issue here is one of transitive dependencies, which is a property on
the original cyclic dependency graph.

So while we have the dependency graph

HsExpr <--> TcRnTypes <--> PmOracle

This is of no issue for build parallelism, because the .hs-boot files
remove "feedback vertices" from the graph, rendering it acyclic:

TcRnTypes.hs-boot <- HsExpr <- TcRnTypes <- PmOracle
PmOracle.hs-boot <- TcRnTypes

Yet a package including HsExpr must also include PmOracle to compile.

The parallelism issue is best tackled by hadrian based profiles, on which
we could compute try to come up with the optimal schedule and compare it
across multiple CI runs.
But the dependency issue here is rather specific to the set of modules that
should be included in ghc-lib-parser, I'm afraid, and I don't see how to
solve it in a general way other than pre-computing the set of transitive
dependencies for specific blessed modules. Which might be a thing we
want... One set for each major entry point to the compiler pipeline, for
example.

Am Di., 17. Sept. 2019 um 10:19 Uhr schrieb Matthew Pickering <
matthewtpickering at gmail.com>:

> This is precisely the problem I was worried about in April.
>
> https://mail.haskell.org/pipermail/ghc-devs/2019-April/017493.html
>
> Any fix should ensure adding a test to make sure it doesn't happen again.
>
> Cheers,
>
> Matt
>
> On Mon, Sep 16, 2019 at 10:38 PM Shayne Fletcher via ghc-devs
> <ghc-devs at haskell.org> wrote:
> >
> > Hi Sebastian,
> >
> > On Mon, Sep 16, 2019 at 5:23 PM Sebastian Graf <sgraf1337 at gmail.com>
> wrote:
> >>
> >> Hi Shayne,
> >>
> >> Sorry to hear that! We didn't consider modularity at all and I would be
> happy to try to refactor in a way that would allow `ghc-lib-parser` to be
> properly separated again.
> >> I'm fairly certain that I didn't directly touch anything parser
> related, but apparently the new cyclic import of PmOracle within TcRnTypes
> (which is also exposed from `ghc-lib-parser`) pulled in the other half of
> GHC.
> >> I'll see if I can fix that tomorrow, if only by extracting a separate
> `Types`-style module.
> >>
> >
> > That sounds awesome. Tremendous. Thank-you! Please feel free to reach
> out to me if there's anything I can do to help your analysis[*]!
> >
> > [*] For the record, the procedure for calculating the `ghc-lib-parser`
> modules is a little complicated by there needing to be some generated
> equivalents of `.hsc` files present for this to work but the procedure is
> at the end of the day just `ghc -M` invoked over `Parser.hs`.
> >
> >>
> >> Cheers,
> >> Sebastian
> >
> >
> > Fingers crossed and all the best!
> >
> > --
> > Shayne Fletcher
> > Language Engineer / +1 917 699 7663
> > Digital Asset, creators of DAML
> >
> > This message, and any attachments, is for the intended recipient(s)
> only, may contain information that is privileged, confidential and/or
> proprietary and subject to important terms and conditions available at
> http://www.digitalasset.com/emaildisclaimer.html. If you are not the
> intended recipient, please delete this
> message._______________________________________________
> > ghc-devs mailing list
> > ghc-devs at haskell.org
> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20190917/6cc60fe2/attachment.html>