On CI

[Ben Gamari]

Simon Peyton Jones via ghc-devs <ghc-devs at haskell.org> writes:

>>   1. Building and testing happen together. When tests failure
>>   spuriously, we also have to rebuild GHC in addition to re-running
>>   the tests. That's pure waste.
>>   https://gitlab.haskell.org/ghc/ghc/-/issues/13897 tracks this more
>>   or less.

> I don't get this.  We have to build GHC before we can test it, don't we?

>> 2 .  We don't cache between jobs.

> This is, I think, the big one.   We endlessly build the exact same binaries.
> There is a problem, though. If we make *any* change in GHC, even a
> trivial refactoring, its binary will change slightly. So now any
> caching build system will assume that anything built by that GHC must
> be rebuilt - we can't use the cached version. That includes all the
> libraries and the stage2 compiler. So caching can save all the
> preliminaries (building the initial Cabal, and large chunk of stage1,
> since they are built with the same bootstrap compiler) but after that
> we are dead.
>
> I don't know any robust way out of this. That small change in the
> source code of GHC might be trivial refactoring, or it might introduce
> a critical mis-compilation which we really want to see in its build
> products.
>
> However, for smoke-testing MRs, on every architecture, we could
> perhaps cut corners. (Leaving Marge to do full diligence.) For
> example, we could declare that if we have the result of compiling
> library module X.hs with the stage1 GHC in the last full commit in
> master, then we can re-use that build product rather than compiling
> X.hs with the MR's slightly modified stage1 GHC. That *might* be
> wrong; but it's usually right.
>
The question is: what happens if the it *is* wrong?

There are three answers here:

 a. Allowing the build pipeline to pass despite a build/test failure
    eliminates most of the benefit of running the job to begin with as
    allow-failure jobs tend to be ignored.

 b. Making the pipeline fail leaves the contributor to pick up the pieces of a
    failure that they may or may not be responsible for, which sounds
    frustrating indeed.

 c. Retry the build, but this time from scratch. This is a tantalizing option
    but carries the risk that we end up doing *more* work than we do now
    (namely, if all jobs end up running both builds)

The only tenable option here in my opinion is (c). It's ugly, but may be
viable.

Cheers,

- Ben

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210219/edec87f1/attachment.sig>