On CI

[John Ericson]

I am also wary of us to deferring checking whole platforms and what not. 
I think that's just kicking the can down the road, and will result in 
more variance and uncertainty. It might be alright for those authoring 
PRs, but it will make Ben's job keeping the system running even more 
grueling.

Before getting into these complex trade-offs, I think we should focus on 
the cornerstone issue that CI isn't incremental.

 1. Building and testing happen together. When tests failure spuriously,
    we also have to rebuild GHC in addition to re-running the tests.
    That's pure waste. https://gitlab.haskell.org/ghc/ghc/-/issues/13897
    tracks this more or less.
 2. We don't cache between jobs. Shake and Make do not enforce
    dependency soundness, nor cache-correctness when the build plan
    itself changes, and this had made this hard/impossible to do safely.
    Naively this only helps with stage 1 and not stage 2, but if we have
    separate stage 1 and --freeze1 stage 2 builds, both can be
    incremental. Yes, this is also lossy, but I only see it leading to
    false failures not false acceptances (if we can also test the stage
    1 one), so I consider it safe. MRs that only work with a slow full
    build because ABI can so indicate.

The second, main part is quite hard to tackle, but I strongly believe 
incrementality is what we need most, and what we should remain focused on.

John

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210218/e70f1525/attachment.html>