How should we treat changes to the GHC API?

[John Ericson]

I would say the current API is....absolutely terrible and so we should 
have no qualms about breaking it. Talking to the people who work on the 
IDE---i.e. the people who are the most adversely impacted by 
churn---they also don't like it and not want it stabilized in its 
current form.

I think the only good long term solution is to modularize GHC into 
multiple packages, giving us conceptually distinct interfaces which 
break at different rates. This is similar to modularizing base as e.g. 
GHC.* breaks far more often than e.g. Data.List.

How we evaluate changes is less important to me as my main priority is 
to get the pace of development much higher so we have a chance on fixing 
all the technical debt to get us out of the current situation. 
Basically, you could count me in "team heat bath"; I think that should 
have the right connotations.

John

On 7/27/20 2:07 PM, Eric Seidel wrote:
> As another former user of the GHC API, I'd say my two biggest complaints were the relative instability of the API and the lack of documentation. I haven't used the API in at least three years though, so it's possible much has changed since my experience.
>
> I remember often having to do significant work to adapt LiquidHaskell to new versions of GHC due to small changes in the API. We often felt like we had to import internal modules (ie other than 'GHC') to get key bits of functionality we needed, which might explain the churn. But it also points to Iavor's point that the public API grew organically and might benefit from a bit of top-down design to make sure it's complete enough for typical use cases.
>
> For documentation, the issue was less the API docs but a lack of "How do I do X?" docs and examples. One problem that I remember being particularly vexing was resolving names in a particular scope (in my case it was always module-level scopes, but I can easily imagine clients that would want to resolve names in local scopes).
>
> I don't know if the API needs to go through something like the Steering Committee, but a stronger focus on API stability and perhaps a broader view of what constitutes (or should be included in) the public-facing API would be welcome!
>
> On Mon, Jul 27, 2020, at 11:45, Iavor Diatchki wrote:
>> In principle, I think we should treat the GHC API like any other
>> library, and try not to break code unnecessarily.  However, my
>> impression is that the GHC API grew somewhat organically, so we may
>> want to put some additional work before we stabilize things too much.
>> It's been a while since I used it, so I might be out of date, but last
>> I looked the GHC API was a module exporting some high-level functions
>> from GHC.   I think that a single module is too small of an API for a
>> project as large as GHC.  In fact, it probably makes sense to define
>> more than one API.  For example, each plugin point should probably have
>> its own API, and that's likely different to the GHC API that exposes
>> functionality such as "load and type check this module here", or "parse
>> and evaluate this string".
>>
>> -Iavor
>>
>>
>>
>> On Mon, Jul 27, 2020 at 4:05 AM Simon Peyton Jones via ghc-devs
>> <ghc-devs at haskell.org> wrote:
>>> What I’m after is a clear opportunity for informed debate, and a clear yes/no decision.  That need not be high overhead.____
>>> __ __
>>> It means paying some upfront cost for design changes.  But that’s better than the legacy cost of dealing with things that turn out, in retrospect, to be less well designed than they could be.____
>>> __ __
>>> We tend to think of APIs as implementation details.  But they are deeply significant, and express key abstractions, just like language designs do.  I think we should treat them just as seriously.____
>>> __ __
>>> Simon____
>>> __ __
>>> *From:* Mathieu Boespflug <m at tweag.io>
>>> *Sent:* 27 July 2020 11:11
>>> *To:* Simon Peyton Jones <simonpj at microsoft.com>
>>> *Cc:* ghc-devs at haskell.org Devs <ghc-devs at haskell.org>
>>> *Subject:* Re: How should we treat changes to the GHC API?____
>>> __ __
>>> I would just point out that decision by committee, and in particular the GHC Proposals process, has a high cost in terms of both total human brain cycles and latency. The cost is entirely justified when it comes to things that are a) hard to revert and b) extremely hard to get right the first time, like new extensions to the language, or c) very sensitive (like governance, say). For things like breaking changes to API's, it's worth writing out what the current problems are. Are users complaining that the API churn is too high? Are they concerned about endemic quality problems with the API?____
>>> __ __
>>> It may be enough to make sure to know who the main users of the API are and tag them as reviewers on these types of changes in GitLab. Or to avoid extra process but enshrine principles that might be necessary to adopt, like saying that existing API functions should always be kept as-is during some deprecation period and new functionality should be exposed in new additions to the API. Principles to be upheld by reviewers.____
>>> __ __
>>> On Mon, Jul 27, 2020 at 10:45:50, Simon Peyton Jones <ghc-devs at haskell.org> wrote:____
>>>> A recent MR for GHC <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.haskell.org%2Fghc%2Fghc%2F-%2Fmerge_requests%2F3758&data=02%7C01%7Csimonpj%40microsoft.com%7C730e52088cb64dcebe1408d8321567e1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637314414803966186&sdata=GCGrNinXtNxPcJfEu%2F%2BifrJJ22BB2bkIy4E9d5IWOuo%3D&reserved=0> (adding machinery for plugins to write data to extensible interface files) made me wonder:  ____
>>>> how we should treat significant changes to the GHC API?____
>>>> Changes to the GHC API, especially to bits used by plugins or by IDEs, are clearly user-visible to an important class of users – they are not just internal to GHC itself.   So, how should we review them?  Should they perhaps be part of the GHC proposals process?  Or some other similar process?   (The collection of experts on the GHC API, plugins, IDEs etc, is rather different to the membership of the GHC steering group.)____
>>>> I'm asking, not to be obstructive, but because the GHC API deserves to be thought of as a whole; in the past it has grown incrementally, without much discussion, and that has not served us well.  But at the moment there is no process, no group to consult.____
>>>> Any views?____
>>>> Simon____
>>>> _______________________________________________
>>>> ghc-devs mailing list
>>>> ghc-devs at haskell.org
>>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs <https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.haskell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-devs&data=02%7C01%7Csimonpj%40microsoft.com%7C730e52088cb64dcebe1408d8321567e1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637314414803966186&sdata=2TwuVzxKm88aevbTooLG3sLeakrSYZziFPNDozFCvHo%3D&reserved=0>____
>>> __ __
>>> _______________________________________________
>>> ghc-devs mailing list
>>> ghc-devs at haskell.org
>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>> _______________________________________________
>> ghc-devs mailing list
>> ghc-devs at haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs