Fwd: Release policies

[Ben Gamari]

"Boespflug, Mathieu" <m at tweag.io> writes:

> ---------- Forwarded message ----------
> From: Boespflug, Mathieu <m at tweag.io>
> Date: 13 December 2017 at 23:03
> Subject: Re: Release policies
> To: Simon Peyton Jones <simonpj at microsoft.com>
> Cc: ghc-devops-group at haskell.org
>
>
> [replying to ghc-devops-group@, which I assume based on your email's
> content is the mailing list you intended.]
>
> Hi Simon,
>
> feedback from downstream consumers of Cabal metadata (e.g. build tool
> authors) will be particularly useful for the discussion here. Here are my
> thoughts as a bystander.
>
> It's worth trying to identify what problems came up during the integer-gmp
> incident in Trac #14558:
>
> * GHC 8.2.1 shipped with integer-gmp-1.0.1.0 but the release notes said
> otherwise.
>
> * GHC 8.2.1 shipped with Cabal-2.0.0.2, but specifically claimed in the
> release notes that cabal-install-1.24 (and by implication any other build
> tool based on Cabal-the-library version 1.24) was supported: "GHC 8.2 only
> works with cabal-install version 1.24 or later. Please upgrade if you have
> an older version of cabal-install."
> 
> * GHC 8.2.2 also claimed Cabal-1.24 support.
>
> * GHC 8.2.1 was released in July 2017 with Cabal-2.0.0.2, a brand new major
> release with breaking changes to the metadata format, without much lead
> time for downstream tooling authors (like Stack) to adapt.
> 
> * But actually if we look at their respective release notes, GHC 8.2.1 was
> relased in July 2017, even though the Cabal website claims that
> Cabal-2.0.0.2 was released in August 2017 (see
> https://www.haskell.org/cabal/download.html). So it looks like GHC didn't
> just not give enough lead time about an upstream dependency it shipped
> with, it shipped with an unreleased version of Cabal!

Perhaps this is true and I admit I wasn't happy about releasing the compiler
without a Cabal release. However, there was no small amount of pressure to
push forward nevertheless as the release was already quite late and the
expectation was a Cabal release would be coming shortly after the GHC
release. Coordination issues like this are a major reason why I think it
would be better if GHC were more decoupled from its dependencies'
upstreams.

I think the approach that we discussed at ICFP, where library
authors must upstream their version bumps before the freeze, just like
any library, is perhaps one way forward although I suspect it exceptions
will need to be made.

> * Libraries that ship with GHC are usually also uploaded to Hackage, to
> make the documentation easily accessible, but integer-gmp-1.0.1.0 was not
> uploaded to Hackage until 4 months after the release.
> 
> * The metadata for integer-gmp-1.0.1.0 as uploaded to Hackage differed from
> the metadata that was actually in the source tarball of GHC-8.2.1 and
> GHC-8.2.2.
>
> * The metadata for integer-gmp-1.0.1.0 as uploaded to Hackage included
> Cabal-2.0 specific syntactic sugar, making the metadata unreadable using
> any tooling that did not link against the Cabal-2.0.0.2 library (or any
> later version).
>
> * It so happened that one particular version of one particular downstream
> build tool, Stack, had a bug, compounding the bad effects of the previous
> point. But a new release has now been made, and in any case that's not a
> problem for GHC to solve. So let's keep that out of the discussion here.
>
> So I suggest we discuss ways to eliminate or reduce the likelihood of any
> of the above problems from occurring again. Here are some ideas:
>
> * GHC should never under any circumstance ship with an unreleased version
> of any independently maintained dependency. Cabal is one such dependency.
> This should hold true for anything else. We could just add that policy to
> the Release Policy.
>
We can adopt this as a policy, but doing so very well may mean that GHC
will be subject to schedule slips beyond its control. We can hope that
upstream maintainers will be responsive, but there is little we can do
when they are not. Of course, if we adopt the policy of disallowing all
but essentially core library bumps during the freeze then we may be able
to mitigate this.

> * Stronger still, GHC should not switch to a new major release of a
> dependency at any time during feature freeze ahead of a release. E.g. if
> Cabal-3.0.0 ships before feature freeze for GHC-9.6, then maybe it's fair
> game to include in GHC. But not if Cabal-3.0.0 hasn't shipped yet.
>
Yes, this I agree with. I think we can be more accomodating of minor
bumps to fix bugs which may come to light during the freeze, but major
releases should be avoided.

> * The 3-release backwards compat rule should apply in all circumstances.
> That means major version bumps of any library GHC ships with, including
> base, should not imply any breaking change in the API's of any such library.
>
I'm not sure I follow what you are suggesting here.

> * GHC does have control over reinstallable packages (like text and
> bytestring): GHC need not ship with the latest versions of these, if indeed
> they introduce breaking changes that would contravene the 3-release policy.
>
> * Note: today, users are effectively tied to whatever version of the
> packages ships with GHC (i.e. the "reinstallable" bit is problematic today
> for various technical reasons). That's why a breaking change in bytestring
> is technically a breaking change in GHC.
>
I don't follow: Only a small fraction of packages, namely those that
explicitly link against the `ghc` library, are tied. Can you clarify
what technical reasons you are referring to here?

> * The current release policy covers API stability, but what about metadata?
> In the extreme, we could say a 3-release policy applies to metadata too.
> Meaning, all metadata shipping with GHC now and in the next 2 releases
> should be parseable by today's version of Cabal and downstream tooling. Is
> such a long lead time necessary? That's for build tool authors to say, and
> a point to negotiate with GHC devs.
>
> * Because there are far fewer consumers of metadata than consumers of say
> base, I think shorter lead time is reasonable. At the other extreme, it
> could even be just the few months during feature freeze.

Right, I wouldn't be opposed to striving for this in principle although
I think we should be aware that breakage is at times necessary and the
policy should accomodate this. I think the important thing is that we be
aware of when we are breaking metadata compatibility and convey this to
our users.

> * The release notes bugs mentioned above and the lack of consistent upload
> to Hackage are a symptom of lack of release automation, I suspect. That's
> how to fix it, but we could also spell out in the Release Policy that GHC
> libraries should all be on Hackage from the day of release.
>
Yes, the hackage uploads have historically been handled manually. I have
and AFAIK most release managers coming before me have generally deferred
this to Herbert as is quite meticulous. However, I think it would be
nice if we could remove the need for human intervention entirely.

> Finally, a question for discussion:
>
> * Hackage allows revising the metadata of an uploaded package even without
> changing the version number. This happens routinely on Hackage today by the
> Hackage trustees. Should this be permitted for packages whose release is
> completely tied to that of GHC itself (like integer-gmp)?
>
If there is a bug in the metadata then I don't think we should
necessarily tie our hands from fixing it. However, we should clearly
take great care when making such changes to avoid further breakage.

Cheers,

- Ben

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20171214/bb76ba2e/attachment.sig>