[GHC DevOps Group] Release policies

[Gershom B]

This proposal seems to have trailed off. I think it is worth putting a bow
on it and getting it signed off on for the GHC release policy page.

Let me restate it:


Bundled library maintainers agree to the following responsibilities:

1) When GHC cuts a feature-freeze branch, they too (if anything has
changed) cut a feature-freeze branch within two weeks at the maximum
(ideally sooner), to be included in the main GHC freeze branch. If
they do not do so, the last released version will be included instead.

2) When GHC releases the first release candidate, maintainers (if
anything has changed) release new versions of their packages, to then
be depended on directly in the GHC repo. All submodules are then
replaced with their proper released versions for GHC release.


I know Chak and I had a back and forth, but I consider it resolved in the
direction of this proposal at this point, as a strict improvement over the
current situation that does not seem to invite further controversy. If in
the event of future things we learn, this proposal is inadequate, we can
revisit then.

The one thing missing from both this proposal and the current release
policy page in general is discussion of the head overlay to hackage. I
think discussion of this should be integrated in both as a separate issue.
In particular — the feature freeze branches of bundled libraries can and
should be regularly updated in the head overlay.

Thoughts?

—Gershom

On December 19, 2017 at 8:53:22 PM, Gershom B (gershomb at gmail.com) wrote:

On December 19, 2017 at 8:15:49 PM, Manuel M T Chakravarty (
chak at justtesting.org) wrote:
> >
> This in particular implies that everything merged to the release
> branch has to have been well tested (the wiki page states, ”[o]nly
> previously agreed on, stable and tested new functionality is
> allowed in.”)

Ok — here’s where I get confused. In my experience, “well tested” is a
relative term. Certainly if we want a three month period between branch and
release, that’s because we’re expecting a _fair amount_ of bugs to shake
out over that period. As you point out, the wiki page says “agreed on,
stable and tested.” But nonetheless — that’s leaving a fair amount of room
for bugs to be found. Since there’s a period when features are written, and
another in which focus is all on bugfixing, then this is to be expected.
Keeping moving the barrier for making it into even a _branch_ higher and
higher seems weird to me. We’re already creating a new set of significant
barriers by even starting to have this branch process at all.

> Implicitly adding new functionality by depending
> on a moving library dependency, which is out of the quality control
> of the GHC team, is not something that I would call ”stable and
> tested functionality”.

And now here’s where you lose me entirely. The point is you wouldn’t be
depending on a moving library dependency. You’d be depending on a _release
branch_ just the same as GHC. Unless you mean to say that we shouldn’t
trust the maintainers of bundled packages to only put “stable and tested”
features in to a release branch? In the abstract, sure. If we were just
including random github repos sourced from stackoverflow answers, sure. But
in the actual concrete history of GHC and Haskell development — we know the
maintainers of these libs. We know they move slowly in general, and they
have processes too. And we know that if these libs, which are hard to
upgrade on their own (because they’re bundled with GHC), are going to get a
wide exposure to not just internal tests, but external users, it will be
via being bundled with GHC.

Since I would expect, based on past experience, that bundled libraries will
often need small bugfixes and tweaks, as you agree, then what you’ve just
sort of proposed is that we release extra versions of them that end up not
bundled with GHC versions, and which are less well tested. What is the
difference between cutting a branch and depending on it, which is what I
propose, and what you propose, except that we’ll end up with extra version
bumps in your scheme, and hackage packages with bugs that haven’t had a
chance to be shaken out, and may, in some cases (integer-gmp, ghc-prim,
base), not even make _sense_ outside of a GHC release.

I understand that its not pleasant, but GHC and the bundled packages have,
for the time being, a very special relationship. We can’t decouple them by
administrative fiat. GHC depends on them, and they also need GHC to run. So
their lifecycles are intertwined, and their development processes are
intertwined. This is how they have been developed and maintained for years.
Why do we suddenly want to pretend that now these maintainers are “out of
the quality control of the GHC team” like they’re foreign entities?

I also want to underline this is not about “new library features” as such —
this is about cross-cutting development of the sort Moritz described that
requires modifying GHC and bundled libs both.

Another element of the two-way character of this relationship is that when
GHC develops a new feature, often it is the bundled libs that are the first
to take advantage of it — and this is a good thing, as it allows further
stress-testing of the feature. (And sometimes, with certain deprication
warnings, etc., the feature is even especially _for_ those libs). But if
the feature turns out to need tweaks in the stability period (and this is
far from unheard of) then it is the bundled libs that need to change to
adapt. If you have already pinned yourself to released versions of libs,
this is very hard!

It would be nice if bundled libs were upstream of GHC, but they’re not, so
we should stop pretending. Everyone’s in the same stream together, and
everyone will have to agree to row :-)

Again — what is objectionable to having a release branch cut instead of a
full release at T-3? What danger from this do you anticipate?

Personally, I see only upside. More flexibility — same mitigation of risk,
same alignment of processes.

We already have a number of changes in the pipeline, not least the new
schedule in any form. How about we try that, along with a more modest
proposal for bundled libs, and just _see how it feels_ before making any
futher decisions. A “big bang” in process doesn’t need to be bigger — that
only introduces more risk factors that will prevent us from evaluating the
first changeset independently. Honestly, before making any further policy
I’d just like to see if one or two releases can be done with the new
schedule at all, at which point people will better be able to judge what
the remaining pain points really are, instead of speculating about other
future problems.

—Gershom
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20180218/6b316ad0/attachment.html>