[ghc-steering-committee] Base library organisation

Simon Peyton Jones simon.peytonjones at gmail.com
Tue Jun 27 15:02:10 UTC 2023 

>
> And I also want to strongly discourage people from using escape hatches
> and rather motivate them to petition the CLC to extend base with a stable
> interface for their needs.
>

I'll all for that!

Therefore, I’d want ghc-internals to be flagged as INTERNAL, GHC itself be
> built with -fpermit-internals, but everyone else having explicitly to opt
> in to that (and for me an easy way to prohibit the use of any
> non-whitelisted packages using -fpermit-internals).
>
I think the question is: how do they "explicitly opt in"?  One possibility
is: they depend on ghc-internals in their .cabal file.  That's explicit,
and it's opting in.

There is some user interface design here, presumably involving cabal.

Simon

On Tue, 27 Jun 2023 at 15:43, Moritz Angermann <moritz.angermann at gmail.com>
wrote:

> I have strong reasons to discourage them. As a consumer of components that
> can transitively depend on any of those, I must make sure none of my
> dependencies depend on these internals (other than GHC itself). Any
> component that depends on ghc-internals is not fit for use in projects I’d
> be responsible for.
>
> I can not continue spending the obscene amounts we spend for compiler
> upgrades.
>
> If someone in a leaf node depends to use internals I have no problem with
> this. If this unintentionally slips into any library that I transitively
> depend on, I have a problem. This needs to be *very* loud and explicit. I
> would almost call it tainted that something depends on APIs that have no
> stability guarantees.
>
> To illustrate this. Let’s assume we have a Haskell application which
> depends on pkg A transitively though 10+ pkgs. pkg A-1.0 decides to depend
> on GHC-internals. Next GHC comes around, breaks GHC-internals, pkg A is now
> broken. pkg A’s master branch is at 3.0, and will now get the update and be
> compatible with GHC-internals. What about 1.0? Touch luck. And now this
> process ripples through 10+ layers of dependencies.
>
> Not only is this exceptionally expensive to adapt to. It also means there
> is no sensible way to measure regressions of the application. The compiler
> upgrade has now caused lots of code changes throughout the whole codebase
> (including dependencies).
>
> Therefore, I’d want ghc-internals to be flagged as INTERNAL, GHC itself be
> built with -fpermit-internals, but everyone else having explicitly to opt
> in to that (and for me an easy way to prohibit the use of any
> non-whitelisted packages using -fpermit-internals).
>
> We want to decouple a stable set of base base libraries from the GHC
> internal libraries that have less rigorous stability guarantees, because
> they are explicitly internal after all. However if we let this leak out too
> easily we have forfeit significant benefits this could bring.
>
> As someone dealing with a massive codebase, how am I justifying spending
> six figures to just make the code compatible with a new compiler and then
> find out that I’m now facing performance regressions?!
>
> Thus form a practical point of view, I want to insulate my codebase as
> much as possible from depending in any form on GHC-internals or other
> libraries with a reduced stability guarantee.
>
> Depending on ghc-internals (or any such reduced stability library) should
> come with high hurdles and excessive bells. (Which could be consciously
> silenced by something like -fpermit-internals).
>
> On the other hand if this is too easy to do, we end up with lots of
> libraries on hackage depending on GHC-internals, and breaking every release
> cycle we’ve won nothing but only complicated things.
>
> Again, we don’t have the facilities for this today. But I would urge us to
> get the asap.
>
> On a final note:
> > If not, and if they have a need, and base doesn't have it, then they
> may want to depend on ghc-internals temporarily and petition CLC to extend
> base.
>
> I consider this a very slippery slope. We want the split to be explicit
> and conscious not for people to end up depending on base and GHC-internal
> because they just need something. It need to be _strongly_ discouraged to
> do so, and it need to be more painful to use than petition the CLC to
> extend base. Otherwise the incentives are aligned in a way that people do
> not petition the CLC but just use GHC-internals.
>
> I do not want to prevent people from this. There are escape hatches. I
> want to to be able to prevent any such software that uses those escape
> hatches. And I also want to strongly discourage people from using escape
> hatches and rather motivate them to petition the CLC to extend base with a
> stable interface for their needs.
>
> Cheers,
>  Moritz
>
> On Tue, 27 Jun 2023 at 4:10 PM, Simon Peyton Jones <
> simon.peytonjones at gmail.com> wrote:
>
>> I don't think we want to flag *individual *bindings in ghc-internals;
>> they are all there for a good, internal, non-deprecated reason.  What we
>> want to discourage is for GHC clients to depend on *any module *in
>> ghc-internals casually, without having a Strong Reason to do so. Why
>> discourage?
>>
>>    - Everything they want should be in base.  And maybe it already is!
>>    If not, and if they have a need, and base doesn't have it, then they may
>>    want to depend on ghc-internals temporarily and petition CLC to extend base.
>>    - If they casually depend on ghc-internals, they impose costs on
>>    themselves (when we change the API) and on us (when they complain about us
>>    changing the API).
>>
>> I have no strong opinion about want form "discourage" takes.  Certainly
>> not "prevent".
>>
>> Simon
>>
>>
>>
>> On Tue, 27 Jun 2023 at 14:59, Moritz Angermann <
>> moritz.angermann at gmail.com> wrote:
>>
>>> I have some concern around the discouraging concept as well.
>>>
>>> Ideally we had some marker INTERNAL like DEPRECATED, that could be
>>> attached to individual bindings, complete modules or even packages. If this
>>> existed, we could have code either error if it used any INTERNAL binding,
>>> and no -fpermit-internals was provided. If this was attached to already
>>> exposed bindings we’d want a warning phase for at least two releases which
>>> I believe DEPRECATED could do?
>>>
>>> However, we don’t have this today, so I wouldn’t want this to block this
>>> proposal from progressing.
>>>
>>> I’m not so much for hiding it. Reading up on it and knowing the details
>>> (or even looking at the source) can be helpful at times. Making discovery
>>> harder by hiding it would not be my preferred route.
>>>
>>> Cheers,
>>>  Moritz
>>>
>>> On Tue, 27 Jun 2023 at 3:09 PM, Arnaud Spiwack <arnaud.spiwack at tweag.io>
>>> wrote:
>>>
>>>> I don't have a strong opinion. I trust the people involved.
>>>>
>>>> The one thing I'll note is that the part about discouraging people from
>>>> depending on ghc-internals seems to involve an awful lot of work. I
>>>> wouldn't count on such work being done promptly. The one thing in the least
>>>> that can be done reasonably cheaply is making sure that every module in
>>>> ghc-internals to be annotated with {-# OPTIONS_HADDOCK hide #-} (so that
>>>> Haddock only generate documentation for the source but not for the module
>>>> interfaces).
>>>>
>>>> /Arnaud
>>>>
>>>> On Tue, 20 Jun 2023 at 09:03, Chris Dornan <chris at chrisdornan.com>
>>>> wrote:
>>>>
>>>>> Really, all LGTM!
>>>>>
>>>>> On 19 Jun 2023, at 22:10, Simon Peyton Jones <
>>>>> simon.peytonjones at gmail.com> wrote:
>>>>>
>>>>> Hello GHC steering committee,
>>>>>
>>>>> Any views about this?
>>>>>
>>>>> Simon
>>>>>
>>>>> On Thu, 15 Jun 2023 at 10:03, Simon Peyton Jones <
>>>>> simon.peytonjones at gmail.com> wrote:
>>>>>
>>>>>> Dear GHC Steering Committee
>>>>>>
>>>>>> Over the last few weeks, Ben Gamari and I have been discussing with
>>>>>> Andrew and Julian from the Core Libraries Committee how to make the Core
>>>>>> Libraries Committee and the GHC developers work together more fluidly; and
>>>>>> that includes the GHC Steering Committee.
>>>>>>
>>>>>> We now have a fairly well fleshed out proposal here.
>>>>>> <https://github.com/Ericson2314/tech-proposals/blob/ghc-base-libraries/proposals/accepted/050-ghc-base-libraries.rst>
>>>>>>
>>>>>> I hope you like it.  As far as this committee is concerned there are
>>>>>> two particular points of note
>>>>>>
>>>>>>    1. We propose a new package, *ghc-experimental*, which depends on
>>>>>>    *base*.  Many GHC proposals involve defining new types and
>>>>>>    functions.  The idea is that these would initially be in
>>>>>>    *ghc-experimental*.  After they stabilise and become widely
>>>>>>    adopted, the author (or anyone else) can make a CLC proposal to move them
>>>>>>    to *base*, which has much stronger stability guarantees.
>>>>>>    2. Section 5.1 suggests a mechanism to involve CLC members in
>>>>>>    proposals that involve new functions and types, at an earlier stage.  Some
>>>>>>    involve *changing *existing types and functions.  It is clearly
>>>>>>    unproductive for us to debate such things at length, and only *then
>>>>>>    *to land it on the CLC.
>>>>>>
>>>>>>    Section 5.1 also suggests that proposals should explicitly (in a
>>>>>>    separate section) call out
>>>>>>
>>>>>>
>>>>>>    - What new types and functions it defines
>>>>>>    - What existing types and functions are changed.
>>>>>>
>>>>>> We should add that to our template.
>>>>>>
>>>>>> At the moment we are just sharing the proposal with relevant
>>>>>> stakeholders (yourselves, CLC, stack folk, cabal folk etc), so that we can
>>>>>> polish any rough edges before making it public.
>>>>>>
>>>>>> So, any views?  Personally I think this is a Big Step Forward.
>>>>>>
>>>>>> Simon
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>> ghc-steering-committee mailing list
>>>>> ghc-steering-committee at haskell.org
>>>>>
>>>>> https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-steering-committee
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> ghc-steering-committee mailing list
>>>>> ghc-steering-committee at haskell.org
>>>>>
>>>>> https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-steering-committee
>>>>>
>>>>
>>>>
>>>> --
>>>> Arnaud Spiwack
>>>> Director, Research at https://moduscreate.com and https://tweag.io.
>>>> _______________________________________________
>>>> ghc-steering-committee mailing list
>>>> ghc-steering-committee at haskell.org
>>>> https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-steering-committee
>>>>
>>> _______________________________________________
>>> ghc-steering-committee mailing list
>>> ghc-steering-committee at haskell.org
>>> https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-steering-committee
>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-steering-committee/attachments/20230627/bb616eee/attachment-0001.html>

More information about the ghc-steering-committee mailing list