[GHC DevOps Group] Release policies
Simon Peyton Jones
simonpj at microsoft.com
Mon Dec 18 11:08:56 UTC 2017
This thread sounds as if it has been productive, though I have not followed the details.
Does anyone feel able to draw it together into a proposed release policy? Along with a summary of the reasoning that led to it?
| -----Original Message-----
| From: Ghc-devops-group [mailto:ghc-devops-group-bounces at haskell.org] On
| Behalf Of Boespflug, Mathieu
| Sent: 18 December 2017 11:03
| To: Manuel M T Chakravarty <manuel.chakravarty at tweag.io>
| Cc: ghc-devops-group at haskell.org; ghc-devs at haskell.org Devs <ghc-
| devs at haskell.org>
| Subject: Re: [GHC DevOps Group] Release policies
| I think Mikhail's point is that if a package says build-type: Simple, then
| we know exactly what its Setup.hs says, and therefore also which part of the
| Cabal API it's using. Easy enough to keep that part stable even if others
| change. Case in point: Cabal-2.0 brought a number of changes to the overall
| API, but nothing that broke calling defaultMain from Distribution.Simple
| (which is what a build-type: Simple script does). At the end of the day the
| consumers of the wider Cabal API are pretty small. A substantial number of
| misc packages on Hackage do it but rarely heavily. Other than that it
| essentially comes down to Stack, cabal-install, Haskell For Mac and... any
| My takeaway from the discussion so far is that the number of heavy consumers
| looks small enough that a draconian BC policy for Cabal-the-library sounds
| overkill, provided, crucially, that everything is in place, by GHC feature
| freeze at the very latest, to allow a smooth migration. A "smooth
| transition" means having a migration guide available before start of feature
| freeze etc, but to Ben's concern stated earlier in this thread (about
| GHC/upstream coupling), ideally also a release.
| I should note that to the extent that GHC tracks upstream releases only (not
| git commits in unknown state), GHC can be released on a timely schedule
| without needing any coordination from upstream maintainers to await new
| releases on their part. So quite apart from the Cabal thing specifically,
| it's worth thinking about asking that the versions of all upstream packages
| only make it into GHC, at the behest of their respective maintainers, after
| a new release of upstream is made. This was already proposed earlier in the
| > * [Proposal:] GHC does not track git commits of upstream dependencies
| > in an unknown state of quality, but tracks vetted and tested releases
| > instead.
| Potentially, this could even mean drastically cutting down on the number of
| git submodules carried in the GHC repo. Since these packages could as well
| be downloaded from Hackage.
| On 18 December 2017 at 05:04, Manuel M T Chakravarty
| <manuel.chakravarty at tweag.io> wrote:
| > Yes, you are right Haskell for Mac also links against Cabal-the-library
| and API changes have regularly required me to fix my code. I guess, I have
| never been particularly stressed about it, because I also link against GHC
| API and that doesn’t even know how to spell API stability — i.e., changes
| required by Cabal are usually drowned out by the chaos inflicted by GHC.
| > In any case, you are making a good point.
| > Mikhail, I don’t understand your response to Mathieu at all. What does the
| build-type have to do with this?
| > Cheers,
| > Manuel
| >> 15.12.2017, 19:41 Boespflug, Mathieu <m at tweag.io>:
| >> Thanks for the feedback, Michael.
| >> Manuel, I believe you are also a Cabal-the-library consumer in Haskell
| For Mac?
| >> Michael, you brought up another problem tangentially related to the
| >> original integer-gmp issue but that was not in my original list
| >> earlier in this thread:
| >> * Cabal-2.0.0 had breaking changes in the API.
| >> This means that by association GHC itself broke BC, because it
| >> shipped with Cabal-2.0, without the usual grace period.
| >> Now, there are far fewer users of Cabal than of base. All, Michael in
| >> his previous email seems to be okay with breaking changes in Cabal
| >> given the conditions he stated (2 months grace period, advance notice
| >> of when the 2 months start). And perhaps this points to the lack of a
| >> need for the regular grace period applying to Cabal. How many other
| >> users of Cabal-the-library are there? In principle, every single
| >> Hackage package out there, which all have a Setup.hs script. Most of
| >> them are trivial, but how many did break because of these API changes?
| >> I for one am pretty happy for Cabal to move fast, but I'm concerned
| >> that these breaking changes happened without any kind of advance
| >> notice. To Simon's original point - there does not to be a clear
| >> policy and a good process surrounding Cabal itself and other GHC
| >> dependencies. So far we discussed mostly metadata changes, not API
| >> changes.
| >> And to be clear, folks did get some (post facto) notice in September:
| >> https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fcoldw
| >> a.st%2Fe%2Fblog%2F2017-09-09-Cabal-2-0.html&data=02%7C01%7Csimonpj%40
| >> microsoft.com%7Ca4ef8935c10c4da1766f08d546070073%7C72f988bf86f141af91
| >> ab2d7cd011db47%7C1%7C0%7C636491918245390697&sdata=YNAuG7XcS9NiRMyklKe
| >> 1LyL6LrxKr%2BlIsu6oxFXb%2Byg%3D&reserved=0. That's helpful, but I
| >> submit that in the future this really should be part of the GHC release
| announcement (which happened over a month before that), and in fact a
| migration guide circulated before the feature freeze, so downstream tooling
| authors can adapt. If this is not possible, then perhaps it's premature for
| GHC to include that given Cabal release.
| >> Again, GHC should always have the option to stick to the old Cabal
| >> version until things get ironed out.
| >> On 15 December 2017 at 08:42, Michael Snoyman <michael at snoyman.com>
| >>> On Thu, Dec 14, 2017 at 12:27 PM, Boespflug, Mathieu <m at tweag.io> wrote:
| >>> [snip]
| >>>> * Or a middle ground: make feature freeze a thing. Meaning that for
| >>>> a couple of months before a major GHC release, the major new Cabal
| >>>> isn't technically released yet, but like GHC itself within this
| >>>> period, it's pretty staid, so not so much a moving target, and
| >>>> something downstream tooling authors can possibly adapt to even
| >>>> without any grace period on new metadata features. This assumes
| >>>> that the 2 months of feature freeze are enough time for downstream
| >>>> tooling. Thoughts from any of those maintainers?
| >>> Short answer: if there's a clear idea in advance of when this
| >>> feature freeze is going to happen, I think we can coordinate
| >>> releases of downstream tooling (Stack being the most important, but
| >>> stackage-curator playing in as well) so that 2 months is sufficient.
| >>> I'll talk with the rest of the Stack team to see if there are any
| >>> Longer answer: Stack intentionally avoids depending on the internals
| >>> of Cabal wherever possible. Instead of calling library functions
| >>> directly from within Haskell code to perform builds, for example, it
| >>> interacts with the Setup.hs files over their command line interface.
| This has two results:
| >>> * Stack can usually start using new GHC/Cabal versions without a new
| >>> Stack release, since it's just shelling out for the actual build
| >>> * There's not usually very much code churn needed in Stack to
| >>> upgrade to a newer Cabal release
| >>> This past release was an exception because of all of the changes
| >>> that landed, both the new cabal grammar to support the ^>= operator
| >>> (making the old parser incapable of lossily parsing new files) and
| >>> API changes (I think mostly around Backpack, though there was some
| >>> code cleanup as well). In particular, the main interface we need
| >>> from Cabal—the package description data types and parser—changed
| >>> significantly enough that it took significant effort to upgrade.
| >>> There were also new features added (like sub libraries and foreign
| >>> libraries) that weren't immediately supported by the old Stack version,
| and had to be manually added in.
| >>> Tying this up: generally upgrading to a new Cabal release should be
| >>> fine, and the only concern I'd have is fitting it into a release
| >>> schedule with Stack. The complications that could slow that down are:
| >>> * Changes to the command line interface that Stack uses (hopefully
| >>> those are exceedingly rare)
| >>> * Major overhauls to the Stack-facing API
| >>> Michael
| >>>  This allows for more reproducible builds of older snapshots,
| >>> insuring that the exact same Cabal library is performing the builds
| >> _______________________________________________
| >> Ghc-devops-group mailing list
| >> Ghc-devops-group at haskell.org
| >> https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devops-group
| Ghc-devops-group mailing list
| Ghc-devops-group at haskell.org
More information about the ghc-devs