[GHC DevOps Group] Continuous integration effort status

[Ben Gamari]

Hi everyone,

Simon suggested that I write with a bit of a status update summarizing
where we stand on the CI reboot effort.

As of today we have the following:

 * Per-push CircleCI builds [1] in a variety of ways:
 
     * "Vanilla" amd64/Linux
     * amd64/Linux with -fllvm
     * Hadrian amd64/Linux
     * "Vanilla" amd64/Darwin

   These currently fail due to #14705, although this should be resolved
   shortly.

 * A 64-bit Windows build of BuildFlavour=quick on Appveyor [2]. This
   currently fails due to a maximum artifact size restriction. Moreover,
   a full GHC build is not practical on Appveyor's infrastructure due to
   build time limits.

To help achieve full Appveyor support, Google X has kindly agreed to
sponsor computational resources which we can use to host our own
instance of Appveyor's self-hosted server. In addition to the cost of
the computational resources which Google has covered, this solution will
require a $50/month Premium Appveyor subscription. As far as I know we
still don't have a source for this, although it is possible that
haskell.org could foot this bill.

There are a variety of items currently in flight,

 * Alp will be looking at fixing the --slow validate failures (#14890)

 * David is looking at incorporating last summer's performance testsuite
   rework, #12758, into our CircleCI infrastructure.

 * For the next day or two I am rather tied up with other things, but
   after this I will have some time to look at the boxes that Greg has
   brought up and see what it will take to bring up Appveyor. If someone
   else would like to pick this up then do let me know.

 * Mark Karpov has done some great work pushing forward some
   patches that I started but didn't have time to finish up. This
   includes,
     * Moving the CircleCI infrastructure to Docker, eliminating a
       spurious testsuite failure [3]

     * Adding a 32-bit Linux target [4]

The following things are rather lacking direction at the moment,

 a. Integration between Phabricator and Appveyor/CircleCI (#14506).

 b. Validation of each commit in a push as both Appveyor and CircleCI
    only build the push head (#14505). This is quite important as having
    a full commit history has been invaluable in identifying
    regressions. I will start pushing commits one-at-a-time until this
    is sorted out.

 c. Artifact preservation. I recommend that we push these to S3 or
    similar to avoid relying on Appveyor/CircleCI's (generally rather
    restrictive) artifact archival facilities.

 d. Hadrian builds seem to non-deterministically hang on CircleCI
    (#14600). I've never observed this locally and have had no luck
    reproducing via SSH on a CircleCI box, so it's unclear what to do
    about this.

 e. Support for building on non-Debian-based systems (e.g. Fedora),
    which is necessary if we want to produce our binary distributions
    via CI

I suspect that after (b) is fixed we will find that the computational
resources of our CircleCI account may be a bit lacking, but we can cross
that bridge once we get there.

Cheers,

- Ben


[1] https://circleci.com/gh/ghc/ghc/tree/master
[2] https://ci.appveyor.com/project/GHCAppveyor/ghc
[3] https://github.com/ghc/ghc/pull/104
[4] https://github.com/ghc/ghc/pull/106
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devops-group/attachments/20180305/93a44bfe/attachment.sig>