What changed between GHC 8.8 and 8.10 that could cause this?

[Christiaan Baaij]

Even if MR388 ( https://gitlab.haskell.org/ghc/ghc/-/merge_requests/388 )
is the cause of the issue we're seeing with the API exposed by Clash, I
still think MR388 is wrong.
My reasoning is the following:

In 8.8 and earlier we had:
- RTS C-code contains the ground truth of what is linked. The API it
provides are set-membership, insert, lookup, and delete. Notably it does
not allow you to get the set of linked objects.
- There is a globally shared MVar (using NOINLINE, sharedCaf,
unsafePerformIO newIORef "tricks") to what is basically a log/view of the
linked-objects state kept by the RTS C-code.

With MR388, in 8.10 and later we get:
- RTS C-code contains the ground truth of what is linked. The API it
provides are set-membership, insert, lookup, and delete. Notably it does
not allow you to get the set of linked objects.
- A _new_ MVar for every call to `runGhc` which is a log/view of the
linked-object state kept by the RTS C-code. But that means these MVar get
out-of-sync with the ground truth that is the RTS C-code! And since the RTS
C-code does not expose an API to get the set of linked objects, there's no
way to sync these MVars either!

I'm building a ghc-8.10.2 with MR388 reverted to see whether it is indeed
what is causing the issue we're seeing in Clash.
Given my analysis above of what I think is wrong with MR388, I'm not saying
we should completely revert MR388, but simply ensure that every HscEnv
created through `runGhc` gets the globally shared MVar; as opposed to the
current call to `newMVar`.

On Sun, 7 Mar 2021 at 04:02, ÉRDI Gergő <gergo at erdi.hu> wrote:

> Thanks Matthew and Julian! Unfortunately, trying out GHC before/after this
> change didn't turn out to be as easy as I hoped: to do my testing, I
> need to build a given GHC commit, and then use that via Stack to install
> ~140 dependencies so that I can then test the problem I have initially
> seen. And it turns out doing that with a random GHC commit is quite
> painful because in any given Stackage snapshot there will be packages with
> which the GHC-bundled libraries are incompatible... :/
>
>
>
> On Thu, 4 Mar 2021, Julian Leviston wrote:
>
> > Hi,I don’t know enough about what Clash does to comment really, but it
> sounds like
> > it’s to do with my work on enabling multiple linker instances
> > in https://gitlab.haskell.org/ghc/ghc/-/merge_requests/388 — maybe
> reading through
> > that or the plan I outlined at
> https://gitlab.haskell.org/ghc/ghc/-/issues/3372 might
> > help, though I’m not sure.
> >
> > Strange, though, as this work was to isolate state in GHC — to change it
> from using a
> > global IORef to use a per-process MVar . But it definitely did change
> the way state is
> > handled, so it might be the related to these issues somehow?
> >
> > I realise this isn’t much help, but maybe it points you in a direction
> where you can
> > begin to understand some more.
> >
> > Julian
> >
> >       On 4 Mar 2021, at 10:55 pm, ÉRDI Gergő <gergo at erdi.hu> wrote:
> >
> > Hi,
> >
> > I'm trying to figure out a Clash  problem and managed to track it down
> to a GHC
> > upgrade; specifically, a given Clash version, when based on GHC 8.8, has
> no
> > problem synthesizing one module after another from one process; but the
> same
> > Clash version with GHC 8.10 fails with link-time errors on the second
> > compilation.
> >
> > The details are at
> https://github.com/clash-lang/clash-compiler/issues/1686
> > but for now I'm just hoping that some lightbulb will go off for someone
> if some
> > handling of internal state has changed in GHC that could mean that the
> symbol
> > tables of loaded modules could persist between GHC invocations from the
> same
> > process.
> >
> > So, does this ring a bell for anyone?
> >
> > Thanks,
> > Gergo
> > _______________________________________________
> > ghc-devs mailing list
> > ghc-devs at haskell.org
> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
> >
> >
> >
> >
>
> --
>
>    .--= ULLA! =-----------------.
>     \     http://gergo.erdi.hu   \
>      `---= gergo at erdi.hu =-------'
> I tried to commit suicide once by taking over 1,000 aspirin. But after I
> took 2, I felt better!_______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210309/e36c13a8/attachment.html>