<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Menlo-Regular;
panose-1:0 0 0 0 0 0 0 0 0 0;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0cm;
margin-right:0cm;
margin-bottom:0cm;
margin-left:36.0pt;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
p.Code, li.Code, div.Code
{mso-style-name:Code;
margin-top:0cm;
margin-right:0cm;
margin-bottom:0cm;
margin-left:36.0pt;
margin-bottom:.0001pt;
font-size:9.0pt;
font-family:"Courier New";}
p.msonormal0, li.msonormal0, div.msonormal0
{mso-style-name:msonormal;
mso-margin-top-alt:auto;
margin-right:0cm;
mso-margin-bottom-alt:auto;
margin-left:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle19
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;
font-weight:normal;
font-style:normal;
text-decoration:none none;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:836961350;
mso-list-type:hybrid;
mso-list-template-ids:982445550 134807553 134807555 134807557 134807553 134807555 134807557 134807553 134807555 134807557;}
@list l0:level1
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Symbol;}
@list l0:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Courier New";}
@list l0:level3
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
@list l0:level4
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Symbol;}
@list l0:level5
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Courier New";}
@list l0:level6
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
@list l0:level7
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Symbol;}
@list l0:level8
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Courier New";}
@list l0:level9
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
@list l1
{mso-list-id:992290824;
mso-list-type:hybrid;
mso-list-template-ids:-888867508 134807553 134807555 134807557 134807553 134807555 134807557 134807553 134807555 134807557;}
@list l1:level1
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Symbol;}
@list l1:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Courier New";}
@list l1:level3
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
@list l1:level4
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Symbol;}
@list l1:level5
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Courier New";}
@list l1:level6
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
@list l1:level7
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Symbol;}
@list l1:level8
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Courier New";}
@list l1:level9
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
@list l2
{mso-list-id:1260337428;
mso-list-type:hybrid;
mso-list-template-ids:-381780212 134807553 134807555 134807557 134807553 134807555 134807557 134807553 134807555 134807557;}
@list l2:level1
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Symbol;}
@list l2:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Courier New";}
@list l2:level3
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
@list l2:level4
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Symbol;}
@list l2:level5
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Courier New";}
@list l2:level6
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
@list l2:level7
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Symbol;}
@list l2:level8
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Courier New";}
@list l2:level9
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
ol
{margin-bottom:0cm;}
ul
{margin-bottom:0cm;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-GB" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal" style="margin-left:36.0pt"><span style="font-size:11.5pt;font-family:"Menlo-Regular",serif">The problem is that many contributors, including Simon PJ, Richard, and<br>
me, tend to push batches of work<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt">I have not been following this thread (“job accounting” seemed above my pay grade) but I saw this mention of my name
</span><span style="font-size:12.0pt;font-family:"Segoe UI Emoji",sans-serif">😊</span><span style="font-size:12.0pt">. Without having read myself into the context there seem to be two issues<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt"><o:p> </o:p></span></p>
<ul style="margin-top:0cm" type="disc">
<li class="MsoListParagraph" style="margin-left:0cm;mso-list:l2 level1 lfo1"><span style="font-size:12.0pt">Every commit to master should be validate-clean, and this should be tested by the CI framework not by the contributor. This is essential. I would be
delighted if every commit I made went through that gate. I’m careful, but occasionally not careful enough.<br>
<br>
<o:p></o:p></span></li><li class="MsoListParagraph" style="margin-left:0cm;mso-list:l2 level1 lfo1"><span style="font-size:12.0pt">Most – perhaps all – commits should go through a code-review process. Here I freely admit that I tend to use (or abuse?) my status to make most of my
commits without review, except perhaps informally with individuals. I’d be absolutely willing to review this if (a) in fact people think that the extra step would really improve quality (perhaps looking at past commits) or (b) the very fact that I do so makes
people feel cross. <o:p></o:p></span></li></ul>
<p class="MsoListParagraph"><span style="font-size:12.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt">In both cases, the big thing from my point of view is that, once I’m ready to press “go”,
<i>I’d like to take it off my to-do list</i> by pushing into a queue that will result in either<o:p></o:p></span></p>
<ul style="margin-top:0cm" type="disc">
<li class="MsoListParagraph" style="margin-left:0cm;mso-list:l0 level1 lfo2"><span style="font-size:12.0pt">a commit to master, or<o:p></o:p></span></li><li class="MsoListParagraph" style="margin-left:0cm;mso-list:l0 level1 lfo2"><span style="font-size:12.0pt">an email to me saying “more work to do here”<o:p></o:p></span></li></ul>
<p class="MsoNormal"><span style="font-size:12.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt">I’m sure that many contributors would value this property. The tricky bit is, I suppose, deciding when the reviewers are happy. Maybe that need central human intervention; I’m not sure.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt">Also important: I often accumulate a sequence of related patches:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt"><o:p> </o:p></span></p>
<ul style="margin-top:0cm" type="disc">
<li class="MsoListParagraph" style="margin-left:0cm;mso-list:l1 level1 lfo3"><span style="font-size:12.0pt">I’d like to be able to queue them as a group, or at least with very low additional overhead per patch<o:p></o:p></span></li></ul>
<p class="MsoListParagraph"><span style="font-size:12.0pt"><o:p> </o:p></span></p>
<ul style="margin-top:0cm" type="disc">
<li class="MsoListParagraph" style="margin-left:0cm;mso-list:l1 level1 lfo3"><span style="font-size:12.0pt">Sometimes I have attempted to separate two things (e.g. a bug fix and a refactoring). In the past I have not felt it important to guarantee a validate-clean
state between the two; e.g. I might have erred in separating the two. The cost benefit ratio of teasing them apart has not seemed worth the pain. But separating them at least means we have two commit messages.
<o:p></o:p></span></li></ul>
<p class="MsoListParagraph"><span style="font-size:12.0pt"><o:p> </o:p></span></p>
<p class="MsoListParagraph"><span style="font-size:12.0pt">On this point I can see that I might have to change my behaviour, because the CI would be obliged to stop as soon as it found one that didn’t validate. I might need help to figure out how best to fix
up a patch sequence by moving little bits from one to another, but I’m sure it’s possible.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt">I’m very keen to support a better CI process in any way I can.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt">Simon<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt"><o:p> </o:p></span></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0cm 0cm 0cm 4.0pt">
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span lang="EN-US">From:</span></b><span lang="EN-US"> Ghc-devops-group [mailto:ghc-devops-group-bounces@haskell.org]
<b>On Behalf Of </b>Manuel M T Chakravarty<br>
<b>Sent:</b> 11 December 2017 06:17<br>
<b>To:</b> Ben Gamari <ben@well-typed.com><br>
<b>Cc:</b> ghc-devops-group@haskell.org; Mateusz Kowalczyk <fuuzetsu@fuuzetsu.co.uk><br>
<b>Subject:</b> Re: [GHC DevOps Group] CircleCI job accounting question<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal">23.11.2017 06:15 schrieb Ben Gamari <<a href="mailto:ben@well-typed.com">ben@well-typed.com</a>>:<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.5pt;font-family:"Menlo-Regular",serif">Simon Marlow <</span><a href="mailto:marlowsd@gmail.com"><span style="font-size:11.5pt;font-family:"Menlo-Regular",serif">marlowsd@gmail.com</span></a><span style="font-size:11.5pt;font-family:"Menlo-Regular",serif">>
writes:<br>
<br style="font-variant-caps: normal;text-align:start;-webkit-text-stroke-width: 0px;word-spacing:0px">
<br>
</span><o:p></o:p></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal"><span style="font-size:11.5pt;font-family:"Menlo-Regular",serif">On 22 November 2017 at 15:31, Ben Gamari <<a href="mailto:ben@well-typed.com">ben@well-typed.com</a>> wrote:<br>
<br>
<br>
<o:p></o:p></span></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal"><span style="font-size:11.5pt;font-family:"Menlo-Regular",serif">Manuel M T Chakravarty <<a href="mailto:manuel.chakravarty@tweag.io">manuel.chakravarty@tweag.io</a>> writes:<br>
<br>
<o:p></o:p></span></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal"><span style="font-size:11.5pt;font-family:"Menlo-Regular",serif">Why do we need the intermediate builds exactly? Wouldn’t they usually<br>
fail? (When I do PRs with multiple commits, the state of the tree<br>
between this commits will usually not be well-defined.)<o:p></o:p></span></p>
</blockquote>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span style="font-size:11.5pt;font-family:"Menlo-Regular",serif"><br>
No, every commit should build. This is in part a difference between<br>
Phabricator's patch-based model and GitHub's feature branch model.<br>
However, many projects using the latter also demand that all<br>
intermediate commits must be atomic, buildable changes. Sacrificing this<br>
property greatly complicates bisection.<br>
<br>
Building all intermediates is desireable as ultimately we would like to<br>
preserve per-commit build artifacts for the last few months of commits<br>
to enable easy bisection.<o:p></o:p></span></p>
</blockquote>
<p class="MsoNormal"><span style="font-size:11.5pt;font-family:"Menlo-Regular",serif"><br>
I don't quite understand this. Yes building all commits is desirable, but<br>
in the case of Phabricator each revision is going to be a single commit,<br>
no? So why is this an issue? Or is it an issue only for github PRs?<o:p></o:p></span></p>
</blockquote>
<p class="MsoNormal"><span style="font-size:11.5pt;font-family:"Menlo-Regular",serif"><br>
The problem is that many contributors, including Simon PJ, Richard, and<br>
me, tend to push batches of work. For instance, when I land<br>
contributors' differentials I first apply a batch, then validate<br>
locally, and then push as a chunk. We can change this if necessary, but<br>
it will need to be via social convention which hasn't worked very well<br>
historically.</span><o:p></o:p></p>
</div>
</blockquote>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
<p class="MsoNormal">I would like to understand this issue (which prompts #14505) a bit better. Do we agree that, in an ideal world, nobody should ever push to master, and instead, all contributions start as PRs or differentials, which are merged after being
reviewed, which crucial includes building them in CI? If that were the case, then we wouldn’t have the issue with needing to force CircleCI to build non-head commits, right?<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">If we agree that this is the ideal scenario, then what is preventing us from getting to that scenario? In particular, I don’t think you should need to validate contributor code manually on your boxes. The purpose of an automatic CI pipeline
is exactly to avoid such manual steps.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I am sorry if any of these questions appear naive.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Cheers,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Manuel<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
</div>
</body>
</html>