<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
span.EmailStyle20
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:1242988014;
mso-list-template-ids:57992096;}
@list l1
{mso-list-id:1457065868;
mso-list-template-ids:57992096;}
@list l2
{mso-list-id:1666782929;
mso-list-template-ids:57992096;}
ol
{margin-bottom:0cm;}
ul
{margin-bottom:0cm;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-GB" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<ol start="1" type="1">
<li class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l2 level1 lfo1">
Building and testing happen together. When tests failure spuriously, we also have to rebuild GHC in addition to re-running the tests. That's pure waste.
<a href="https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.haskell.org%2Fghc%2Fghc%2F-%2Fissues%2F13897&data=04%7C01%7Csimonpj%40microsoft.com%7C3d503922473f4cd0543f08d8d48522b2%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637493018301253098%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=FG2fyYCXbacp69Q8Il6GE0aX%2B7ZLNkH1u84NA%2FVMjQc%3D&reserved=0">
https://gitlab.haskell.org/ghc/ghc/-/issues/13897</a> tracks this more or less.<o:p></o:p></li></ol>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US">I don’t get this. We have to build GHC before we can test it, don’t we?<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:18.0pt">
2 . We don't cache between jobs. <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">This is, I think, the big one. We endlessly build the exact same binaries.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">There is a problem, though. If we make *<b>any</b>* change in GHC, even a trivial refactoring, its binary will change slightly. So now any caching build system will assume that
anything built by that GHC must be rebuilt – we can’t use the cached version. That includes all the libraries and the stage2 compiler. So caching can save all the preliminaries (building the initial Cabal, and large chunk of stage1, since they are built
with the same bootstrap compiler) but after that we are dead.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">I don’t know any robust way out of this. That small change in the source code of GHC might be trivial refactoring, or it might introduce a critical mis-compilation which we really
want to see in its build products. <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">However, for smoke-testing MRs, on every architecture, we could perhaps cut corners. (Leaving Marge to do full diligence.) For example, we could declare that if we have the result
of compiling library module X.hs with the stage1 GHC in the last full commit in master, then we can re-use that build product rather than compiling X.hs with the MR’s slightly modified stage1 GHC. That *<b>might</b>* be wrong; but it’s usually right.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Anyway, there are big wins to be had here.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Simon<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:18.0pt">
<o:p> </o:p></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0cm 0cm 0cm 4.0pt">
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span lang="EN-US">From:</span></b><span lang="EN-US"> ghc-devs <ghc-devs-bounces@haskell.org>
<b>On Behalf Of </b>John Ericson<br>
<b>Sent:</b> 19 February 2021 03:19<br>
<b>To:</b> ghc-devs <ghc-devs@haskell.org><br>
<b>Subject:</b> Re: On CI<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p>I am also wary of us to deferring checking whole platforms and what not. I think that's just kicking the can down the road, and will result in more variance and uncertainty. It might be alright for those authoring PRs, but it will make Ben's job keeping
the system running even more grueling.<o:p></o:p></p>
<p>Before getting into these complex trade-offs, I think we should focus on the cornerstone issue that CI isn't incremental.<o:p></o:p></p>
<ol start="1" type="1">
<li class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l0 level1 lfo3">
Building and testing happen together. When tests failure spuriously, we also have to rebuild GHC in addition to re-running the tests. That's pure waste.
<a href="https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.haskell.org%2Fghc%2Fghc%2F-%2Fissues%2F13897&data=04%7C01%7Csimonpj%40microsoft.com%7C3d503922473f4cd0543f08d8d48522b2%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637493018301253098%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=FG2fyYCXbacp69Q8Il6GE0aX%2B7ZLNkH1u84NA%2FVMjQc%3D&reserved=0">
https://gitlab.haskell.org/ghc/ghc/-/issues/13897</a> tracks this more or less.<o:p></o:p></li><li class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l0 level1 lfo3">
We don't cache between jobs. Shake and Make do not enforce dependency soundness, nor cache-correctness when the build plan itself changes, and this had made this hard/impossible to do safely. Naively this only helps with stage 1 and not stage 2, but if we have
separate stage 1 and --freeze1 stage 2 builds, both can be incremental. Yes, this is also lossy, but I only see it leading to false failures not false acceptances (if we can also test the stage 1 one), so I consider it safe. MRs that only work with a slow
full build because ABI can so indicate.<o:p></o:p></li></ol>
<p class="MsoNormal">The second, main part is quite hard to tackle, but I strongly believe incrementality is what we need most, and what we should remain focused on.<o:p></o:p></p>
<p>John<o:p></o:p></p>
</div>
</div>
</body>
</html>