Reinstallable - base

Simon Peyton Jones simon.peytonjones at gmail.com
Fri Oct 20 08:00:22 UTC 2023 

>
> A very large proportion of libraries, and virtually all end-user
> applications, transitively depend on Template Haskell. Whether they use
> Template Haskell directly or not. So if we're saying “base is
> reinstallable, except when you have Template Haskell somewhere”, we're
> effectively saying “base is not reinstallable”. Now, it could be a good
> stepping-stone, from an engineering standpoint, but I don't think we could
> deliver this and be satisfied that we've accomplished anything.
>

No one has yet answered my naive question (from 3 days ago) asking why
Template Haskell stops base being reinstallable.  I'll quote it here for
completeness.

Let's say that
>

   - An old library mylib (which uses TH) depends on base-4.7.
   - A new GHC, say GHC 9.10, depends on a newer version of base-4.9, which
   in turn depends on ghc-internal-9.10.
   - At the same time, though, we release base-4.7.1, which depends on
   ghc-internal-9.10, and exposes the base-4.7 API.

At this point we use ghc-9.10 to compile L, against base-4.7.1.   (Note
that the ghc-9.10 binary includes a compiled form of `base-4.9`.)


   - That produces compiled object files, such as, mylib:M.o.
   - To run TH we need to link them with the running binary
   - So we need to link the compiled `base-4.7.1` as well.  No problem: it
   contains very little code; it is mostly a shim for ghc-internal-9.10

So the only thing we need is the ability to have a single linked binary
that includes (the compiled form for) two different versions/instantiations
of `base`.   I think that's already supported: each has a distinct
"installed package id".

(End of quote)

What am I missing?

Simon

On Fri, 20 Oct 2023 at 08:57, Arnaud Spiwack <arnaud.spiwack at tweag.io>
wrote:

> A very large proportion of libraries, and virtually all end-user
> applications, transitively depend on Template Haskell. Whether they use
> Template Haskell directly or not. So if we're saying “base is
> reinstallable, except when you have Template Haskell somewhere”, we're
> effectively saying “base is not reinstallable”. Now, it could be a good
> stepping-stone, from an engineering standpoint, but I don't think we could
> deliver this and be satisfied that we've accomplished anything.
>
> On Thu, 19 Oct 2023 at 13:47, Oleg Grenrus <oleg.grenrus at iki.fi> wrote:
>
>> For what it worth, `template-haskell` itself depends on a `base`. So if
>> `base` if different base is used, different `template-haskell` is to be
>> used.
>>
>> In my opinion is not *too unfair* to require that if you actually splice
>> in (i.e. the code not only provides template-haskell combinators to
>> create/modify splices) then you must have base and template-haskell
>> versions aligned with host GHC used versions.
>>
>> The same restriction is GHC plugins, isn't it, except `template-haskell`
>> is replaced with `ghc`?
>>
>> - Oleg
>>
>> On 17.10.2023 18.54, Adam Gundry wrote:
>> > Hi Simon,
>> >
>> > Thanks for starting this discussion, it would be good to see progress
>> > in this direction. As it happens I was discussing this question with
>> > Ben and Matt over dinner last night, and unfortunately they explained
>> > to me that it is more difficult than I naively hoped, even once
>> > wired-in and known-key things are moved to ghc-internal.
>> >
>> > The difficulty is that, as a normal Haskell library, ghc itself will
>> > be compiled against a particular version of base. Then when Template
>> > Haskell is used (with the internal interpreter), code will be
>> > dynamically loaded into a process that already has symbols for ghc's
>> > version of base, which means it is not safe for the code to depend on
>> > a different version of base. This is rather like the situation with TH
>> > and cross-compilers.
>> >
>> > Adam
>> >
>> >
>> >
>> > On 17/10/2023 11:08, Simon Peyton Jones wrote:
>> >> Dear GHC devs
>> >>
>> >> Given the now-agreed split between ghc-internal and base
>> >> <https://github.com/haskellfoundation/tech-proposals/pull/51>, what
>> >> stands in the way of a "reinstallable base"?
>> >>
>> >> Specifically, suppose that
>> >>
>> >>   * GHC 9.8 comes out with base-4.9
>> >>   * The CLC decides to make some change to `base`, so we get base-4.10
>> >>   * Then GHC 9.10 comes out with base-4.10
>> >>
>> >> I think we'd all like it if someone could use GHC 9.10 to compile a
>> >> library L that depends on base-4.9 and either L doesn't work at all
>> >> with base-4.10, or L's dependency bounds have not yet been adjusted
>> >> to allow base-4.10.
>> >>
>> >> We'd like to have a version of `base`, say `base-4.9.1` that has the
>> >> exact same API as `base-4.9` but works with GHC 9.10.
>> >>
>> >> Today, GHC 9.10 comes with a specific version of base, /and you can't
>> >> change it/. The original reason for that was, I recall, that GHC
>> >> knows the precise place where (say) the type Int is declared, and
>> >> it'll get very confused if that data type definition moves around.
>> >>
>> >> But now we have `ghc-internal`, all these "things that GHC magically
>> >> knows" are in `ghc-internal`, not `base`.
>> >>
>> >> *Hence my question: what (now) stops us making `base` behave like any
>> >> other library*?  That would be a big step forward, because it would
>> >> mean that a newer GHC could compile old libraries against their old
>> >> dependencies.
>> >>
>> >> (Some changes would still be difficult.  If, for example, we removed
>> >> Monad and replaced it with classes Mo1 and Mo2, it might be hard to
>> >> simulate the old `base` with a shim.  But getting 99% of the way
>> >> there would still be fantastic.)
>> >>
>> >> Simon
>> >
>> _______________________________________________
>> ghc-devs mailing list
>> ghc-devs at haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>
>
>
> --
> Arnaud Spiwack
> Director, Research at https://moduscreate.com and https://tweag.io.
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20231020/d87df749/attachment.html>

More information about the ghc-devs mailing list