[GHC DevOps Group] CircleCI job accounting question

[Ben Gamari]

Manuel M T Chakravarty <manuel.chakravarty at tweag.io> writes:

>> Ben Gamari <ben at well-typed.com>:
>> 
>> Manuel M T Chakravarty <manuel.chakravarty at tweag.io> writes:
>> 
>>> Mateusz had a first stab
>>> 
>>>  https://github.com/tweag/ghc/blob/tweag/circleci-macos/appveyor.yml
>>> 
>>> but got stuck in the default resource limits. We emailed them with a
>>> request, but there was no answer so far. I’ll follow up on it.
>>> 
>> Any update on this? For the record, I have confirmed with the Rustaceans
>> that Mozilla indeed pays for their usage.
>
> No, sorry, I have been completely taken out with travelling and
> conference for the last week. (Just arrived in the Netherlands.)

Quite alright; just wondering.

>> * It appears that CircleCI only builds the head commits of pushes.
>>   Making this configurable has been a feature request for nearly a year
>>   now, so it looks like we will need to work around this. I briefly
>>   looked into setting up some automation to trigger builds on otherwise
>>   untested commits, but ran into apparent API bugginess. It looks like
>>   we'll just need to ensure that contributors push at most one commit
>>   at a time for now to ensure all commits get testing. See GHC #14505
>>   for details.
>
> Why do we need the intermediate builds exactly? Wouldn’t they usually
> fail? (When I do PRs with multiple commits, the state of the tree
> between this commits will usually not be well-defined.)

No, every commit should build. This is in part a difference between
Phabricator's patch-based model and GitHub's feature branch model.
However, many projects using the latter also demand that all
intermediate commits must be atomic, buildable changes. Sacrificing this
property greatly complicates bisection.

Building all intermediates is desireable as ultimately we would like to
preserve per-commit build artifacts for the last few months of commits
to enable easy bisection.

>> * I have tried enabling testing of Harbormaster Differentials via
>>   CircleCI. Unfortunately it appears that CircleCI only supports
>>   testing repositories hosted on GitHub. There are a few ways in which
>>   we could proceed,
>> 
>>    a. Move ghc's staging area (the repository where Arcanist pushes
>>       patches submitted with `arc diff`) to GitHub. This, however, would
>>       require that we manually manage push privileges to this repository.
>
> What do you mean by manually manage push privileges? In what way is
> that not manually at the moment?

As long as a user had added a key to their Phabricator account pushing
to the staging area will "just work". It requires no intervention from
me. I believe in the event that we moved to GitHub I would need to
manually grant users commit privileges to the staging area.

>>    b. Try to work around the issue by mirroring GHC's staging area to
>>       GitHub and manually trigger CircleCI builds.
>
> Is the manual triggering necessary, because Harbormaster would need to
> wait until the repo is triggered (which it can’t)?

In general this whole mirroring situation doesn't appear to be a
use-case that Phabricator's CircleCI integration supports. It demands
that the staging area of the tested repository is hosted on GitHub.

>> * I have been honing the Hadrian test infrastructure; I'm currently
>>   waiting on a build, but I expect this attempt will pass, at which point
>>   I will merge it.
>
> Great!

Sadly the build appears to reliably hang with no output. I'll need to
look into this.

Cheers,

- Ben
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devops-group/attachments/20171122/0aefba88/attachment.sig>