[GHC DevOps Group] Continuous integration effort status

[Manuel M T Chakravarty]

Thanks, Ben. That is a very helpful summary.

Manuel

> 06.03.2018 04:15 Ben Gamari <ben at well-typed.com>:
> 
> Hi everyone,
> 
> Simon suggested that I write with a bit of a status update summarizing
> where we stand on the CI reboot effort.
> 
> As of today we have the following:
> 
> * Per-push CircleCI builds [1] in a variety of ways:
> 
>     * "Vanilla" amd64/Linux
>     * amd64/Linux with -fllvm
>     * Hadrian amd64/Linux
>     * "Vanilla" amd64/Darwin
> 
>   These currently fail due to #14705, although this should be resolved
>   shortly.
> 
> * A 64-bit Windows build of BuildFlavour=quick on Appveyor [2]. This
>   currently fails due to a maximum artifact size restriction. Moreover,
>   a full GHC build is not practical on Appveyor's infrastructure due to
>   build time limits.
> 
> To help achieve full Appveyor support, Google X has kindly agreed to
> sponsor computational resources which we can use to host our own
> instance of Appveyor's self-hosted server. In addition to the cost of
> the computational resources which Google has covered, this solution will
> require a $50/month Premium Appveyor subscription. As far as I know we
> still don't have a source for this, although it is possible that
> haskell.org could foot this bill.
> 
> There are a variety of items currently in flight,
> 
> * Alp will be looking at fixing the --slow validate failures (#14890)
> 
> * David is looking at incorporating last summer's performance testsuite
>   rework, #12758, into our CircleCI infrastructure.
> 
> * For the next day or two I am rather tied up with other things, but
>   after this I will have some time to look at the boxes that Greg has
>   brought up and see what it will take to bring up Appveyor. If someone
>   else would like to pick this up then do let me know.
> 
> * Mark Karpov has done some great work pushing forward some
>   patches that I started but didn't have time to finish up. This
>   includes,
>     * Moving the CircleCI infrastructure to Docker, eliminating a
>       spurious testsuite failure [3]
> 
>     * Adding a 32-bit Linux target [4]
> 
> The following things are rather lacking direction at the moment,
> 
> a. Integration between Phabricator and Appveyor/CircleCI (#14506).
> 
> b. Validation of each commit in a push as both Appveyor and CircleCI
>    only build the push head (#14505). This is quite important as having
>    a full commit history has been invaluable in identifying
>    regressions. I will start pushing commits one-at-a-time until this
>    is sorted out.
> 
> c. Artifact preservation. I recommend that we push these to S3 or
>    similar to avoid relying on Appveyor/CircleCI's (generally rather
>    restrictive) artifact archival facilities.
> 
> d. Hadrian builds seem to non-deterministically hang on CircleCI
>    (#14600). I've never observed this locally and have had no luck
>    reproducing via SSH on a CircleCI box, so it's unclear what to do
>    about this.
> 
> e. Support for building on non-Debian-based systems (e.g. Fedora),
>    which is necessary if we want to produce our binary distributions
>    via CI
> 
> I suspect that after (b) is fixed we will find that the computational
> resources of our CircleCI account may be a bit lacking, but we can cross
> that bridge once we get there.
> 
> Cheers,
> 
> - Ben
> 
> 
> [1] https://circleci.com/gh/ghc/ghc/tree/master
> [2] https://ci.appveyor.com/project/GHCAppveyor/ghc
> [3] https://github.com/ghc/ghc/pull/104
> [4] https://github.com/ghc/ghc/pull/106
> _______________________________________________
> Ghc-devops-group mailing list
> Ghc-devops-group at haskell.org
> https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devops-group