[GHC DevOps Group] Release policies

Mon Dec 18 11:03:22 UTC 2017

I think Mikhail's point is that if a package says build-type: Simple,
then we know exactly what its Setup.hs says, and therefore also which
part of the Cabal API it's using. Easy enough to keep that part stable
even if others change. Case in point: Cabal-2.0 brought a number of
changes to the overall API, but nothing that broke calling defaultMain
from Distribution.Simple (which is what a build-type: Simple script
does). At the end of the day the consumers of the wider Cabal API are
pretty small. A substantial number of misc packages on Hackage do it
but rarely heavily. Other than that it essentially comes down to
Stack, cabal-install, Haskell For Mac and... any others?

My takeaway from the discussion so far is that the number of heavy
consumers looks small enough that a draconian BC policy for
Cabal-the-library sounds overkill, provided, crucially, that
everything is in place, by GHC feature freeze at the very latest, to
allow a smooth migration. A "smooth transition" means having a
migration guide available before start of feature freeze etc, but to
Ben's concern stated earlier in this thread (about GHC/upstream
coupling), ideally also a release.

I should note that to the extent that GHC tracks upstream releases
only (not git commits in unknown state), GHC can be released on a
timely schedule without needing any coordination from upstream
maintainers to await new releases on their part. So quite apart from
the Cabal thing specifically, it's worth thinking about asking that
the versions of all upstream packages only make it into GHC, at the
behest of their respective maintainers, after a new release of
upstream is made. This was already proposed earlier in the thread:

> * [Proposal:] GHC does not track git commits of upstream dependencies
> in an unknown state of quality, but tracks vetted and tested releases
> instead.

Potentially, this could even mean drastically cutting down on the
number of git submodules carried in the GHC repo. Since these packages
could as well be downloaded from Hackage.

On 18 December 2017 at 05:04, Manuel M T Chakravarty
<manuel.chakravarty at tweag.io> wrote:
> Yes, you are right Haskell for Mac also links against Cabal-the-library and API changes have regularly required me to fix my code. I guess, I have never been particularly stressed about it, because I also link against GHC API and that doesn’t even know how to spell API stability — i.e., changes required by Cabal are usually drowned out by the chaos inflicted by GHC.
>
> In any case, you are making a good point.
>
> Mikhail, I don’t understand your response to Mathieu at all. What does the build-type have to do with this?
>
> Cheers,
> Manuel
>
>> 15.12.2017, 19:41 Boespflug, Mathieu <m at tweag.io>:
>>
>> Thanks for the feedback, Michael.
>>
>> Manuel, I believe you are also a Cabal-the-library consumer in Haskell For Mac?
>>
>> Michael, you brought up another problem tangentially related to the
>> original integer-gmp issue but that was not in my original list
>> earlier in this thread:
>>
>> * Cabal-2.0.0 had breaking changes in the API.
>>
>> This means that by association GHC itself broke BC, because it shipped
>> with Cabal-2.0, without the usual grace period.
>>
>> Now, there are far fewer users of Cabal than of base. All, Michael in
>> his previous email seems to be okay with breaking changes in Cabal
>> given the conditions he stated (2 months grace period, advance notice
>> of when the 2 months start). And perhaps this points to the lack of a
>> need for the regular grace period applying to Cabal. How many other
>> users of Cabal-the-library are there? In principle, every single
>> Hackage package out there, which all have a Setup.hs script. Most of
>> them are trivial, but how many did break because of these API changes?
>> I for one am pretty happy for Cabal to move fast, but I'm concerned
>> that these breaking changes happened without any kind of advance
>> notice. To Simon's original point - there does not to be a clear
>> policy and a good process surrounding Cabal itself and other GHC
>> dependencies. So far we discussed mostly metadata changes, not API
>> changes.
>>
>> And to be clear, folks did get some (post facto) notice in September:
>> http://coldwa.st/e/blog/2017-09-09-Cabal-2-0.html. That's helpful, but
>> I submit that in the future this really should be part of the GHC
>> release announcement (which happened over a month before that), and in
>> fact a migration guide circulated before the feature freeze, so
>> downstream tooling authors can adapt. If this is not possible, then
>> perhaps it's premature for GHC to include that given Cabal release.
>> Again, GHC should always have the option to stick to the old Cabal
>> version until things get ironed out.
>>
>>
>> On 15 December 2017 at 08:42, Michael Snoyman <michael at snoyman.com> wrote:
>>>
>>>
>>> On Thu, Dec 14, 2017 at 12:27 PM, Boespflug, Mathieu <m at tweag.io> wrote:
>>>
>>> [snip]
>>>
>>>> * Or a middle ground: make feature freeze a thing. Meaning that for a
>>>> couple of months before a major GHC release, the major new Cabal isn't
>>>> technically released yet, but like GHC itself within this period, it's
>>>> pretty staid, so not so much a moving target, and something downstream
>>>> tooling authors can possibly adapt to even without any grace period on
>>>> new metadata features. This assumes that the 2 months of feature
>>>> freeze are enough time for downstream tooling. Thoughts from any of
>>>> those maintainers?
>>>>
>>>
>>> Short answer: if there's a clear idea in advance of when this feature freeze
>>> is going to happen, I think we can coordinate releases of downstream tooling
>>> (Stack being the most important, but stackage-curator playing in as well) so
>>> that 2 months is sufficient. I'll talk with the rest of the Stack team to
>>> see if there are any concerns.
>>>
>>> Longer answer: Stack intentionally avoids depending on the internals of
>>> Cabal wherever possible. Instead of calling library functions directly from
>>> within Haskell code to perform builds, for example, it interacts with the
>>> Setup.hs files over their command line interface.[1] This has two results:
>>>
>>> * Stack can usually start using new GHC/Cabal versions without a new Stack
>>> release, since it's just shelling out for the actual build
>>> * There's not usually very much code churn needed in Stack to upgrade to a
>>> newer Cabal release
>>>
>>> This past release was an exception because of all of the changes that
>>> landed, both the new cabal grammar to support the ^>= operator (making the
>>> old parser incapable of lossily parsing new files) and API changes (I think
>>> mostly around Backpack, though there was some code cleanup as well). In
>>> particular, the main interface we need from Cabal—the package description
>>> data types and parser—changed significantly enough that it took significant
>>> effort to upgrade. There were also new features added (like sub libraries
>>> and foreign libraries) that weren't immediately supported by the old Stack
>>> version, and had to be manually added in.
>>>
>>> Tying this up: generally upgrading to a new Cabal release should be fine,
>>> and the only concern I'd have is fitting it into a release schedule with
>>> Stack. The complications that could slow that down are:
>>>
>>> * Changes to the command line interface that Stack uses (hopefully those are
>>> exceedingly rare)
>>> * Major overhauls to the Stack-facing API
>>>
>>> Michael
>>>
>>> [1] This allows for more reproducible builds of older snapshots, insuring
>>> that the exact same Cabal library is performing the builds
>> _______________________________________________
>> Ghc-devops-group mailing list
>> Ghc-devops-group at haskell.org
>> https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devops-group
>