[ghc-steering-committee] Base library organisation

Tue Jun 27 14:43:11 UTC 2023

I have strong reasons to discourage them. As a consumer of components that
can transitively depend on any of those, I must make sure none of my
dependencies depend on these internals (other than GHC itself). Any
component that depends on ghc-internals is not fit for use in projects I’d
be responsible for.

I can not continue spending the obscene amounts we spend for compiler
upgrades.

If someone in a leaf node depends to use internals I have no problem with
this. If this unintentionally slips into any library that I transitively
depend on, I have a problem. This needs to be *very* loud and explicit. I
would almost call it tainted that something depends on APIs that have no
stability guarantees.

To illustrate this. Let’s assume we have a Haskell application which
depends on pkg A transitively though 10+ pkgs. pkg A-1.0 decides to depend
on GHC-internals. Next GHC comes around, breaks GHC-internals, pkg A is now
broken. pkg A’s master branch is at 3.0, and will now get the update and be
compatible with GHC-internals. What about 1.0? Touch luck. And now this
process ripples through 10+ layers of dependencies.

Not only is this exceptionally expensive to adapt to. It also means there
is no sensible way to measure regressions of the application. The compiler
upgrade has now caused lots of code changes throughout the whole codebase
(including dependencies).

Therefore, I’d want ghc-internals to be flagged as INTERNAL, GHC itself be
built with -fpermit-internals, but everyone else having explicitly to opt
in to that (and for me an easy way to prohibit the use of any
non-whitelisted packages using -fpermit-internals).

We want to decouple a stable set of base base libraries from the GHC
internal libraries that have less rigorous stability guarantees, because
they are explicitly internal after all. However if we let this leak out too
easily we have forfeit significant benefits this could bring.

As someone dealing with a massive codebase, how am I justifying spending
six figures to just make the code compatible with a new compiler and then
find out that I’m now facing performance regressions?!

Thus form a practical point of view, I want to insulate my codebase as much
as possible from depending in any form on GHC-internals or other libraries
with a reduced stability guarantee.

Depending on ghc-internals (or any such reduced stability library) should
come with high hurdles and excessive bells. (Which could be consciously
silenced by something like -fpermit-internals).

On the other hand if this is too easy to do, we end up with lots of
libraries on hackage depending on GHC-internals, and breaking every release
cycle we’ve won nothing but only complicated things.

Again, we don’t have the facilities for this today. But I would urge us to
get the asap.

On a final note:
> If not, and if they have a need, and base doesn't have it, then they may
want to depend on ghc-internals temporarily and petition CLC to extend base.

I consider this a very slippery slope. We want the split to be explicit and
conscious not for people to end up depending on base and GHC-internal
because they just need something. It need to be _strongly_ discouraged to
do so, and it need to be more painful to use than petition the CLC to
extend base. Otherwise the incentives are aligned in a way that people do
not petition the CLC but just use GHC-internals.

I do not want to prevent people from this. There are escape hatches. I want
to to be able to prevent any such software that uses those escape hatches.
And I also want to strongly discourage people from using escape hatches and
rather motivate them to petition the CLC to extend base with a stable
interface for their needs.

Cheers,
 Moritz

On Tue, 27 Jun 2023 at 4:10 PM, Simon Peyton Jones <
simon.peytonjones at gmail.com> wrote:

> I don't think we want to flag *individual *bindings in ghc-internals;
> they are all there for a good, internal, non-deprecated reason.  What we
> want to discourage is for GHC clients to depend on *any module *in
> ghc-internals casually, without having a Strong Reason to do so. Why
> discourage?
>
>    - Everything they want should be in base.  And maybe it already is!
>    If not, and if they have a need, and base doesn't have it, then they may
>    want to depend on ghc-internals temporarily and petition CLC to extend base.
>    - If they casually depend on ghc-internals, they impose costs on
>    themselves (when we change the API) and on us (when they complain about us
>    changing the API).
>
> I have no strong opinion about want form "discourage" takes.  Certainly
> not "prevent".
>
> Simon
>
>
>
> On Tue, 27 Jun 2023 at 14:59, Moritz Angermann <moritz.angermann at gmail.com>
> wrote:
>
>> I have some concern around the discouraging concept as well.
>>
>> Ideally we had some marker INTERNAL like DEPRECATED, that could be
>> attached to individual bindings, complete modules or even packages. If this
>> existed, we could have code either error if it used any INTERNAL binding,
>> and no -fpermit-internals was provided. If this was attached to already
>> exposed bindings we’d want a warning phase for at least two releases which
>> I believe DEPRECATED could do?
>>
>> However, we don’t have this today, so I wouldn’t want this to block this
>> proposal from progressing.
>>
>> I’m not so much for hiding it. Reading up on it and knowing the details
>> (or even looking at the source) can be helpful at times. Making discovery
>> harder by hiding it would not be my preferred route.
>>
>> Cheers,
>>  Moritz
>>
>> On Tue, 27 Jun 2023 at 3:09 PM, Arnaud Spiwack <arnaud.spiwack at tweag.io>
>> wrote:
>>
>>> I don't have a strong opinion. I trust the people involved.
>>>
>>> The one thing I'll note is that the part about discouraging people from
>>> depending on ghc-internals seems to involve an awful lot of work. I
>>> wouldn't count on such work being done promptly. The one thing in the least
>>> that can be done reasonably cheaply is making sure that every module in
>>> ghc-internals to be annotated with {-# OPTIONS_HADDOCK hide #-} (so that
>>> Haddock only generate documentation for the source but not for the module
>>> interfaces).
>>>
>>> /Arnaud
>>>
>>> On Tue, 20 Jun 2023 at 09:03, Chris Dornan <chris at chrisdornan.com>
>>> wrote:
>>>
>>>> Really, all LGTM!
>>>>
>>>> On 19 Jun 2023, at 22:10, Simon Peyton Jones <
>>>> simon.peytonjones at gmail.com> wrote:
>>>>
>>>> Hello GHC steering committee,
>>>>
>>>> Any views about this?
>>>>
>>>> Simon
>>>>
>>>> On Thu, 15 Jun 2023 at 10:03, Simon Peyton Jones <
>>>> simon.peytonjones at gmail.com> wrote:
>>>>
>>>>> Dear GHC Steering Committee
>>>>>
>>>>> Over the last few weeks, Ben Gamari and I have been discussing with
>>>>> Andrew and Julian from the Core Libraries Committee how to make the Core
>>>>> Libraries Committee and the GHC developers work together more fluidly; and
>>>>> that includes the GHC Steering Committee.
>>>>>
>>>>> We now have a fairly well fleshed out proposal here.
>>>>> <https://github.com/Ericson2314/tech-proposals/blob/ghc-base-libraries/proposals/accepted/050-ghc-base-libraries.rst>
>>>>>
>>>>> I hope you like it.  As far as this committee is concerned there are
>>>>> two particular points of note
>>>>>
>>>>>    1. We propose a new package, *ghc-experimental*, which depends on
>>>>>    *base*.  Many GHC proposals involve defining new types and
>>>>>    functions.  The idea is that these would initially be in
>>>>>    *ghc-experimental*.  After they stabilise and become widely
>>>>>    adopted, the author (or anyone else) can make a CLC proposal to move them
>>>>>    to *base*, which has much stronger stability guarantees.
>>>>>    2. Section 5.1 suggests a mechanism to involve CLC members in
>>>>>    proposals that involve new functions and types, at an earlier stage.  Some
>>>>>    involve *changing *existing types and functions.  It is clearly
>>>>>    unproductive for us to debate such things at length, and only *then
>>>>>    *to land it on the CLC.
>>>>>
>>>>>    Section 5.1 also suggests that proposals should explicitly (in a
>>>>>    separate section) call out
>>>>>
>>>>>
>>>>>    - What new types and functions it defines
>>>>>    - What existing types and functions are changed.
>>>>>
>>>>> We should add that to our template.
>>>>>
>>>>> At the moment we are just sharing the proposal with relevant
>>>>> stakeholders (yourselves, CLC, stack folk, cabal folk etc), so that we can
>>>>> polish any rough edges before making it public.
>>>>>
>>>>> So, any views?  Personally I think this is a Big Step Forward.
>>>>>
>>>>> Simon
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>> ghc-steering-committee mailing list
>>>> ghc-steering-committee at haskell.org
>>>> https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-steering-committee
>>>>
>>>>
>>>> _______________________________________________
>>>> ghc-steering-committee mailing list
>>>> ghc-steering-committee at haskell.org
>>>> https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-steering-committee
>>>>
>>>
>>>
>>> --
>>> Arnaud Spiwack
>>> Director, Research at https://moduscreate.com and https://tweag.io.
>>> _______________________________________________
>>> ghc-steering-committee mailing list
>>> ghc-steering-committee at haskell.org
>>> https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-steering-committee
>>>
>> _______________________________________________
>> ghc-steering-committee mailing list
>> ghc-steering-committee at haskell.org
>> https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-steering-committee
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-steering-committee/attachments/20230627/d0ac501f/attachment-0001.html>