Reinstallable - base

Wed Aug 7 15:37:41 UTC 2024

Thanks for writing this up!

> Note: should template-haskell be merged into base to make dependencies
simpler?

I think the answer is "no" because it has non-base dependencies. We could
vendor all of these like we do for the containers and filepath dependencies
ATM, but I'm not sure if it's worth it.

Cheers,
Teo

On Wed, 7 Aug 2024, 16:27 Sylvain Henry, <sylvain at haskus.fr> wrote:

> Hi Simon,
>
> As this came up again during the GHC call yesterday, I've written some
> notes to explain the issues and the current status (as I understand them):
> https://hsyl20.fr/posts/2024-08-07-about-ghcs-stability.html
>
> I hope that helps. Happy to get some feedback about this by other people
> involved in the process.
>
> Sylvain
>
>
> On 20/10/2023 10:00, Simon Peyton Jones wrote:
>
> A very large proportion of libraries, and virtually all end-user
>> applications, transitively depend on Template Haskell. Whether they use
>> Template Haskell directly or not. So if we're saying “base is
>> reinstallable, except when you have Template Haskell somewhere”, we're
>> effectively saying “base is not reinstallable”. Now, it could be a good
>> stepping-stone, from an engineering standpoint, but I don't think we could
>> deliver this and be satisfied that we've accomplished anything.
>>
>
> No one has yet answered my naive question (from 3 days ago) asking why
> Template Haskell stops base being reinstallable.  I'll quote it here for
> completeness.
>
> Let's say that
>>
>
>    - An old library mylib (which uses TH) depends on base-4.7.
>    - A new GHC, say GHC 9.10, depends on a newer version of base-4.9,
>    which in turn depends on ghc-internal-9.10.
>    - At the same time, though, we release base-4.7.1, which depends on
>    ghc-internal-9.10, and exposes the base-4.7 API.
>
> At this point we use ghc-9.10 to compile L, against base-4.7.1.   (Note
> that the ghc-9.10 binary includes a compiled form of `base-4.9`.)
>
>
>    - That produces compiled object files, such as, mylib:M.o.
>    - To run TH we need to link them with the running binary
>    - So we need to link the compiled `base-4.7.1` as well.  No problem:
>    it contains very little code; it is mostly a shim for ghc-internal-9.10
>
> So the only thing we need is the ability to have a single linked binary
> that includes (the compiled form for) two different versions/instantiations
> of `base`.   I think that's already supported: each has a distinct
> "installed package id".
>
> (End of quote)
>
> What am I missing?
>
> Simon
>
> On Fri, 20 Oct 2023 at 08:57, Arnaud Spiwack <arnaud.spiwack at tweag.io>
> wrote:
>
>> A very large proportion of libraries, and virtually all end-user
>> applications, transitively depend on Template Haskell. Whether they use
>> Template Haskell directly or not. So if we're saying “base is
>> reinstallable, except when you have Template Haskell somewhere”, we're
>> effectively saying “base is not reinstallable”. Now, it could be a good
>> stepping-stone, from an engineering standpoint, but I don't think we could
>> deliver this and be satisfied that we've accomplished anything.
>>
>> On Thu, 19 Oct 2023 at 13:47, Oleg Grenrus <oleg.grenrus at iki.fi> wrote:
>>
>>> For what it worth, `template-haskell` itself depends on a `base`. So if
>>> `base` if different base is used, different `template-haskell` is to be
>>> used.
>>>
>>> In my opinion is not *too unfair* to require that if you actually splice
>>> in (i.e. the code not only provides template-haskell combinators to
>>> create/modify splices) then you must have base and template-haskell
>>> versions aligned with host GHC used versions.
>>>
>>> The same restriction is GHC plugins, isn't it, except `template-haskell`
>>> is replaced with `ghc`?
>>>
>>> - Oleg
>>>
>>> On 17.10.2023 18.54, Adam Gundry wrote:
>>> > Hi Simon,
>>> >
>>> > Thanks for starting this discussion, it would be good to see progress
>>> > in this direction. As it happens I was discussing this question with
>>> > Ben and Matt over dinner last night, and unfortunately they explained
>>> > to me that it is more difficult than I naively hoped, even once
>>> > wired-in and known-key things are moved to ghc-internal.
>>> >
>>> > The difficulty is that, as a normal Haskell library, ghc itself will
>>> > be compiled against a particular version of base. Then when Template
>>> > Haskell is used (with the internal interpreter), code will be
>>> > dynamically loaded into a process that already has symbols for ghc's
>>> > version of base, which means it is not safe for the code to depend on
>>> > a different version of base. This is rather like the situation with TH
>>> > and cross-compilers.
>>> >
>>> > Adam
>>> >
>>> >
>>> >
>>> > On 17/10/2023 11:08, Simon Peyton Jones wrote:
>>> >> Dear GHC devs
>>> >>
>>> >> Given the now-agreed split between ghc-internal and base
>>> >> <https://github.com/haskellfoundation/tech-proposals/pull/51>, what
>>> >> stands in the way of a "reinstallable base"?
>>> >>
>>> >> Specifically, suppose that
>>> >>
>>> >>   * GHC 9.8 comes out with base-4.9
>>> >>   * The CLC decides to make some change to `base`, so we get base-4.10
>>> >>   * Then GHC 9.10 comes out with base-4.10
>>> >>
>>> >> I think we'd all like it if someone could use GHC 9.10 to compile a
>>> >> library L that depends on base-4.9 and either L doesn't work at all
>>> >> with base-4.10, or L's dependency bounds have not yet been adjusted
>>> >> to allow base-4.10.
>>> >>
>>> >> We'd like to have a version of `base`, say `base-4.9.1` that has the
>>> >> exact same API as `base-4.9` but works with GHC 9.10.
>>> >>
>>> >> Today, GHC 9.10 comes with a specific version of base, /and you can't
>>> >> change it/. The original reason for that was, I recall, that GHC
>>> >> knows the precise place where (say) the type Int is declared, and
>>> >> it'll get very confused if that data type definition moves around.
>>> >>
>>> >> But now we have `ghc-internal`, all these "things that GHC magically
>>> >> knows" are in `ghc-internal`, not `base`.
>>> >>
>>> >> *Hence my question: what (now) stops us making `base` behave like any
>>> >> other library*?  That would be a big step forward, because it would
>>> >> mean that a newer GHC could compile old libraries against their old
>>> >> dependencies.
>>> >>
>>> >> (Some changes would still be difficult.  If, for example, we removed
>>> >> Monad and replaced it with classes Mo1 and Mo2, it might be hard to
>>> >> simulate the old `base` with a shim.  But getting 99% of the way
>>> >> there would still be fantastic.)
>>> >>
>>> >> Simon
>>> >
>>> _______________________________________________
>>> ghc-devs mailing list
>>> ghc-devs at haskell.org
>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>>
>>
>>
>> --
>> Arnaud Spiwack
>> Director, Research at https://moduscreate.com and https://tweag.io.
>> _______________________________________________
>> ghc-devs mailing list
>> ghc-devs at haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>
>
> _______________________________________________
> ghc-devs mailing listghc-devs at haskell.orghttp://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20240807/02346099/attachment.html>