more releases

[Ben Gamari]

Richard Eisenberg <eir at cis.upenn.edu> writes:

> On Sep 1, 2015, at 12:01 AM, Herbert Valerio Riedel <hvriedel at gmail.com> wrote:
>
>> I'd say mostly organisational overhead which can't be fully automated
>> (afaik, Ben has already automated large parts but not everything can be):
>> 
>> - Coordinating with people creating and testing the bindists
>
> This was the sort of thing I thought could be automated. I'm picturing
> a system where Austin/Ben hits a button and everything whirs to life,
> creating, testing, and posting bindists, with no people involved.
>
I can nearly do this for Linux with my existing tools. I can do 32- and
64-bit builds for both RedHat and Debian all on a single
Debian 8 machine with the tools I developed during the course of the
7.10.2 release [1].

Windows is unfortunately still a challenge. I did the 7.10.2 builds on
an EC2 instance and the experience wasn't terribly fun. I would love for
this to be further automated but I've not done this yet.

>> - Writing releases notes & announcment
>
> Release notes should, theoretically, be updated with the patches.
> Announcement can be automated.
>
If I'm doing my job well the release notes shouldn't be a problem. I've
been trying to be meticulous about ensuring that all new features come
with acceptable release notes.

>> - If bundled core-libraries are affected, coordination overhead with package
>>   maintainers (unless GHC HQ owned), verifying version bumps (API diff!) and
>>   changelogs have been updated accordingly, uploading to Hackage
>
> Any library version change would require a more proper release. Do
> these libraries tend to change during a major release cycle?
> 
The core libraries are perhaps the trickiest part of this. Currently the
process goes something like this,

  1. We branch off a stable GHC release
  2. Development continues on `master`, eventually a breaking change is
     merged to one of the libraries
  3. Eventually someone notices and bumps the library's version
  4. More breaking changes are merged to the library
  5. We branch off for another stable release, right before the release
     we manually push the libraries to Hackage
  6. Repeat from (2)

There can potentially be a lot of interface churn between steps 3 and 5.
If we did releases in this period we would need to be much more careful
about library versioning. I suspect this may end up being quite a bit of
work to do properly.

Technically we could punt on this problem and just do the same sort of
stable/unstable versioning for the libraries that we already do with GHC
itself. This would mean, however, that we couldn't upload the libraries
to Hackage.

>> - Uploading and signing packagees to download.haskell.org, and verifying
>>   the downloads 
>
> This isn't automated?
>
It is now (see [2]). This shouldn't be a problem.

>> Austin & Ben probably have more to add to this list
>> 
> I'm sure they do.
>
> Again, I'd be fine if the answer from the community is "it's just not
> what we need". But I wanted to see if there were
> technical/practical/social reasons why this was or wasn't a good idea.
> If we do think it's a good idea absent those reasons, then we can work
> on addressing those concerns.
>
Technically I think there are no reasons why this isn't feasible with
some investment. Exactly how much investment depends upon what
exactly we want to achieve,

 * How often do we make these releases?
 * Which platforms do we support?
 * How carefully do we version included libraries?

If we focus solely on Linux and punt on the library versioning issue I
would say this wouldn't even difficult. I could easily setup my build
machine to do a nightly bindist and push it to a server somewhere.
Austin has also mentioned that Harbormaster builds could potentially
produce bindists.

The question is whether users want more rapid releases. Those working on
GHC will use their own builds. Most users want something reasonably
stable (in both the interface sense and the reliability sense) and
therefore I suspect would stick with the releases. This leaves a
relatively small number of potential users; namely those who want to
play around with unreleased features yet aren't willing to do their own
builds.

Cheers,

- Ben


[1] https://github.com/bgamari/ghc-utils
[2] https://github.com/bgamari/ghc-utils/blob/master/rel-eng/upload.sh
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 472 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20150902/1fd63c41/attachment.sig>