Tentative high-level plans for 7.10.1

Tue Oct 7 06:43:06 UTC 2014

Hello,

Note: you actually don't have to backport anything. Leave it for people
how are interested in LTS release.

As haskell enthusiast, I like all the features GHC comes with each
release. But as working haskell programmer I'm tired. All my code I
wrote at work will probably work with ghc-6.8, but I have to switch to
newer ghc twice a year. (The last time it was because gcc/clang issue on
mac os)

LTS release means you MAY backport fixes. If you want or have time, if
there are people interested in that, etc.
Probably we'll have more chances that hackage libraries will support LTS
releases longer then they support regular releases now. As a result it
will be easer to introduce breaking changes like AMP or
Traversable/Foldable proposal.

Thanks,
Yuras

On Mon, 2014-10-06 at 19:45 -0500, Austin Seipp wrote:
> The steps for making a GHC release are here:
> https://ghc.haskell.org/trac/ghc/wiki/MakingReleases
> 
> So, for the record, making a release is not *that* arduous, but it
> does take time. On average it will take me about 1 day or so to go
> from absolutely-nothing to release announcement:
> 
>  1. Bump version, update configure.ac, tag.
>  2. Build source tarball (this requires 1 build, but can be done very quickly).
>  3. Make N binary builds for each platform (the most time consuming
> part, as this requires heavy optimizations in the builds).
>  4. Upload documentation for all libraries.
>  5. Update webpage and upload binaries.
>  6. Send announcement.
>  7. Upload binaries from other systems later.
> 
> Herbert has graciously begun taking care of stewarding and uploading
> the libraries. So, there are a few steps we could introduce to
> alleviate this process technically in a few ways, but ultimately all
> of these have to happen, pretty much (regardless of the automation
> involved).
> 
> But I don't think this is the real problem.
> 
> The real problem is that GHC moves forward in terms of implementation
> extremely, extremely quickly. It is not clear how to reconcile this
> development pace with something like needing dozens of LTS releases
> for a stable version. At least, not without a lot of concentrated
> effort from almost every single developer. A lot of it can be
> alleviated through social process perhaps, but it's not strictly
> technical IMO.
> 
> What do I mean by that? I mean that:
> 
>  - We may introduce a feature in GHC version X.Y
>  - That might have a bug, or other problems.
>  - We may fix it, and in the process, fix up a few other things and
> refactor HEAD, which will be GHC X.Y+2 eventually.
>  - Repeat steps 2-3 a few times.
>  - Now we want to backport the fixes for that feature in HEAD back to X.Y.
>  - But GHC X.Y has *significantly* diverged from HEAD in that
> timeframe, because of step 3 being repeated!
> 
> In other words: we are often so aggressive at refactoring code that
> the *act* of backporting in and of itself can be complicated, and it
> gets harder as time goes on - because often the GHC of a year ago is
> so much different than the GHC of today.
> 
> As a concrete example of this, let's look at the changes between GHC
> 7.8.2 and GHC 7.8.3:
> 
> https://github.com/ghc/ghc/compare/ghc-7.8.2-release...ghc-7.8.3-release
> 
> There are about ~110 commits between 7.8.2 and 7.8.3. But as the 7.8
> branch lived on, backporting fixes became significantly more complex.
> In fact, I estimate close to 30 of those commits were NOT direct 7.8
> requirements - but they were brought in because _actual fixes_ were
> dependent on them, in non-trivial ways.
> 
> Take for example f895f33 by Simon PJ, which fixes #9023. The problem
> with f895f33 is that by the time we fixed the bug in HEAD with that
> commit, the history had changed significantly from the branch. In
> order to get f895f33 to plant easily, I had to backport *at least* 12
> to 15 other commits, which it was dependent upon, and commits those
> commits were dependent upon, etc etc. I did not see any non-trivial
> way to do this otherwise.
> 
> I believe at one point Gergo backported some of his fixes to 7.8,
> which had since become 'non applicable' (and I thank him for that
> greatly), but inevitably we instead brought along the few extra
> changes anyway, since they were *still* needed for other fixes. And
> some of them had API changes. So the choice was to rewrite 4 patches
> for an old codebase completely (the work being done by two separate
> people) or backport a few extra patches.
> 
> The above is obviously an extreme case. But it stands to reason this
> would _only happen again_ with 7.8.4, probably even worse since more
> months of development have gone by.
> 
> An LTS release would mandate things like no-API-changes-at-all, but
> this significantly limits our ability to *actually* backport patches
> sometimes, like the above, due to dependent changes. The alternative,
> obviously, is to do what Gergo did and manually re-write such a fix
> for the older branch. But that means we would have had to do that for
> *every patch* in the same boat, including 2 or 3 other fixes we
> needed!
> 
> Furthermore, while I am a release manager and do think I know a bit
> about GHC, it is hopeless to expect me to know it all. I will
> absolutely require coordinated effort to help develop 'retropatches'
> that don't break API compatibility, from active developers who are
> involved in their respective features. And they are almost all
> volunteers! Simon and I are the only ones who wouldn't qualify on
> that.
> 
> So - at what point does it stop becoming 'backporting fixes to older
> versions' and instead become literally "working on the older version
> of the compiler AND the new one in tandem"? Given our rate of churn
> and change internally, this seems like it would be a significant
> burden in general to ask of developers. If we had an LTS release of
> GHC that lasted 3 years for example, that would mean developers are
> expected to work on the current code of their own, *and their old code
> for the next three years*. That is an absolutely, undeniably a _huge_
> investment to ask of someone. It's not clear how many can actually
> hold it (and I don't blame them).
> 
> This email is already a bit long (which is extremely unusual for my
> emails, I'm sure you all know), but I just wanted to give some insight
> on the process.
> 
> I think the technical/automation aspects are the easy part. We could
> probably fully automate the GHC release process in days, if one or two
> people worked on it dilligently. The hard part is actually balancing
> the needs and time of users and developers, which is a complex
> relationship.
> 
> On Mon, Oct 6, 2014 at 6:22 PM, John Lato <jwlato at gmail.com> wrote:
> > On Mon, Oct 6, 2014 at 5:38 PM, Johan Tibell <johan.tibell at gmail.com> wrote:
> >>
> >> On Mon, Oct 6, 2014 at 11:28 AM, Herbert Valerio Riedel
> >> <hvriedel at gmail.com> wrote:
> >>>
> >>> On 2014-10-06 at 11:03:19 +0200, p.k.f.holzenspies at utwente.nl wrote:
> >>> > The danger, of course, is that people aren't very enthusiastic about
> >>> > bug-fixing older versions of a compiler, but for
> >>> > language/compiler-uptake, this might actually be a Better Way.
> >>>
> >>> Maybe some of the commercial GHC users might be interested in donating
> >>> the manpower to maintain older GHC versions. It's mostly a
> >>> time-consuming QA & auditing process to maintain old GHCs.
> >>
> >>
> >> What can we do to make that process cheaper? In particular, which are the
> >> manual steps in making a new GHC release today?
> >
> >
> > I would very much like to know this as well.  For ghc-7.8.3 there were a
> > number of people volunteering manpower to finish up the release, but to the
> > best of my knowledge those offers weren't taken up, which makes me think
> > that the extra overhead for coordinating more people would outweigh any
> > gains.  From the outside, it appears that the process/workflow could use
> > some improvement, perhaps in ways that would make it simpler to divide up
> > the workload.
> >
> > John L.
> >
> >
> > _______________________________________________
> > ghc-devs mailing list
> > ghc-devs at haskell.org
> > http://www.haskell.org/mailman/listinfo/ghc-devs
> >
> 
> 
> 