Continuous Integration and Cross Compilation

[William Knop]

Hi Pali and all,

Sorry for the delayed replies; a bunch of things came up and I probably won’t be able to respond properly for two days or so. I am very interested in progressing with this as soon as I can. Many apologies!

Will


On Jun 20, 2014, at 6:15 AM, Páli Gábor János <pali.gabor at gmail.com> wrote:

> Hello William,
> 
> 2014-06-20 0:50 GMT+02:00 William Knop <william.knop.nospam at gmail.com>:
>> 1. We have a pretty good spread of buildbots, but as far as I know there aren’t
>> very many of them. Running only the test suite would increase their utility by
>> roughly 5x (from looking at the buildbot time breakdowns [1]).
> 
> How would this increase their utility?  I naively believe the purpose
> of CI is to rebuild and test the source code after each changeset to
> see if it was bringing regressions.  Running the test suite only does
> not seem to convey this.  Many of the regressions could be observed
> build-time, which means the most safe bet would be to rebuild and test
> everything on the very same platform.
> 
>> 2. Building ghc is time and resource intensive, which makes it hard for people
>> to host buildbots. Even though my machines are relatively new, I can’t usually
>> host one because it would interfere with my other work. I would be more
>> tempted to if it was limited to just the test suite, and perhaps others would as
>> well.
> 
> My buildbots complete the steps (git clone, full build, testing) in
> about 1 hour 40 minutes (with about 1 hour 15 minutes spent in the
> compilation phase), while they run in parallel with a shift about an
> hour.  They run on the same machine, together with the coordination
> server.  This is just a 3.4-GHz 4-core Intel Core i5, with a couple of
> GBs of RAM, I would not call it a high-end box, though.
> 
> Note that it is on purpose that the builders do not use -j for builds,
> meaning that they do not parallelize the invoked make(1)-subprocesses,
> which automatically makes the builds longer.  Perhaps it would be
> worth experimenting with incremental builds and allowing for parallel
> builds as they could cut down on the build times more efficiently.