Reinstallable - base

[Ben Gamari]

Simon Peyton Jones <simon.peytonjones at gmail.com> writes:

> Dear GHC devs
>
> Given the now-agreed split between ghc-internal and base
> <https://github.com/haskellfoundation/tech-proposals/pull/51>, what stands
> in the way of a "reinstallable base"?
>
> Specifically, suppose that
>
>    - GHC 9.8 comes out with base-4.9
>    - The CLC decides to make some change to `base`, so we get base-4.10
>    - Then GHC 9.10 comes out with base-4.10
>
We thought about this quite a bit at the Well-Typed meeting this week.
In short, I suspect that after the `ghc-internal` split is merged and
some further work we will be able to use multiple `base` versions with
a single compiler version.

I imagine this would be done in roughly three phases. The first two
phases can happen in the short term; the last is a bit further off.
I'll describe these in turn below.


# Phase 1: Reinstallable `base`

Specifically, (in the absence of TH) we need to...

 * Give `base` a proper unit ID. This is necessary as GHC may allow
   multiple versions of a single package in a single build; under the
   status quo this would result in overlapping symbol names.

 * Ensure that all known-key and wired-in declarations are placed in
   `ghc-internal`, not `base`. This is necessary as wired-in and
   known-key declarations are detected by their `Name`, which we cannot
   know a priori if `base` has a proper unit ID.

 * Teach `Cabal` that `base` is a reinstallable package (`ghc-internal`
   will remain non-reinstallable)

 * With each GHC version, provide minor releases of `base` which can be
   built against the new GHC's `ghc-internal`

With this done, we can offer (limited) compatibility across GHC/`base`
version pairs. Naturally, changes in semantics of things defined in
`ghc-internal` will make general compatibility difficult or impossible.
However, Simon suggests, limited compatibility is much better than the
status quo.


# Phase 2: Template Haskell

The above story is made considerably messier by `template-haskell`.
Specifically, `template-haskell` depends upon `base`. Typically (i.e.
when not using the external interpreter) values (e.g., splice results)
are communicated between the splice and the compiler as heap objects;
since these heap objects may be of types defined in `base` we must
guarantee that `template-haskell` and `ghc` are built against the same
`base` ABI. Since GHC doesn't have a notion of ABI stability, this would
render reinstallable `base` unusable for a majority of programs.

The first step to fixing this would be to eliminate the ABI dependency
between `template-haskell` and `ghc`. We can do this by either:

 * using the external interpreter (`-fexternal-interpreter`), in which
   case splice results are communicated by serialisation

 * teaching the internal interpreter to communicate using the same
   serialisation mechanism used in the external interpreter

However, `template-haskell` suffers from the same problem as `base` in
that it contains definitions of known-key declarations. For this reason,
we must apply the same approach that we used in `base` above, moving all
known-key definitions into a new `template-haskell-internal` package
against which `template-haskell` can link.

At this point we can teach `Cabal` that `template-haskell` can be
reinstalled and users will be able to build packages using
`TemplateHaskell` while linking against arbitrary `base` versions.


# Phase 3: Compatibility across `template-haskell` versions

One of the major compatibility headaches of moving between GHC versions
is accommodating changes in the `template-haskell` package. While there
are various interface-level changes which we can use to alleviate some
of this pressure (#20828, #24021), being able to provide compatibility
with multiple `template-haskell` versions would be very convenient.

Once we have split `template-haskell-internal` and `template-haskell`,
we can potentially use a similar approach to that proposed above:
support for multiple `template-haskell` major versions can be provided
for each compiler.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20231020/8ef172e2/attachment.sig>