From manuel.chakravarty at tweag.io Fri Nov 3 03:58:22 2017 From: manuel.chakravarty at tweag.io (Manuel M T Chakravarty) Date: Fri, 3 Nov 2017 14:58:22 +1100 Subject: [GHC DevOps Group] Merge request Message-ID: <61B54DBB-A35E-42F3-B810-44969A169709@tweag.io> Hi Ben, We are at a point, where we think, it would be useful to merge the work on CircleCI integration into HEAD: https://ghc.haskell.org/trac/ghc/ticket/14416 I would also like to suggest to enable the per-commit builds on GitHub once the PR has been merged. I’d be happy to help set this up if somebody gives me the relevant access permissions. Cheers, Manuel From manuel.chakravarty at tweag.io Fri Nov 3 04:26:20 2017 From: manuel.chakravarty at tweag.io (Manuel M T Chakravarty) Date: Fri, 3 Nov 2017 15:26:20 +1100 Subject: [GHC DevOps Group] CI status Message-ID: <94B46A63-2C95-4E5F-B81B-AA9F93170443@tweag.io> Hi all! It has now been two weeks since we decided to use CircleCI & AppVeyor as CI solutions for GHC going forward. Hence, I’d like to give you an update on the status of this effort. On the basis of Mathieu’s original CI integration for Tweag I/O’s CircleCI build on our linear types fork of GHC, Mateusz Kowalczyk (a Tweag engineer) completed the Linux and macOS build, fast validation, and storage of the binary distribution (for now, as a temporary solution, in CircleCI’s artefact store). As you may have seen in my previous email, this work is now ready to be merged. Ben and Mateusz also discussed a variation of these scheme that uses source and binary distributions instead of in-tree validation to perform the regression checking. This would have the advantage of also testing the substantial build system infrastructure that creates distribution tarballs. However, due to a number of problems (tickets #14392, #14411 & #14412) this is currently not possible. We still want to eventually move to this more comprehensive scheme, but for now we are moving this of the critical path to get the initial system running. Mateusz and I contacted AppVeyor concerning a more generous than the standard limit for OSS projects (for the Windows build), but haven’t heard back yet. In addition to merging the work so far (#14416), the next steps are * support for PR builds and integration with Phabricator and * Windows support. BTW, you can always track the status at https://ghc.haskell.org/trac/ghc/wiki/ContinuousIntegration Cheers, Manuel From ben at well-typed.com Fri Nov 3 18:00:12 2017 From: ben at well-typed.com (Ben Gamari) Date: Fri, 03 Nov 2017 14:00:12 -0400 Subject: [GHC DevOps Group] CI status In-Reply-To: <94B46A63-2C95-4E5F-B81B-AA9F93170443@tweag.io> References: <94B46A63-2C95-4E5F-B81B-AA9F93170443@tweag.io> Message-ID: <87mv438cpf.fsf@ben-laptop.smart-cactus.org> Manuel M T Chakravarty writes: > Hi all! > > It has now been two weeks since we decided to use CircleCI & AppVeyor as CI solutions for GHC going forward. Hence, I’d like to give you an update on the status of this effort. > > On the basis of Mathieu’s original CI integration for Tweag I/O’s CircleCI build on our linear types fork of GHC, Mateusz Kowalczyk (a Tweag engineer) completed the Linux and macOS build, fast validation, and storage of the binary distribution (for now, as a temporary solution, in CircleCI’s artefact store). As you may have seen in my previous email, this work is now ready to be merged. > > Ben and Mateusz also discussed a variation of these scheme that uses source and binary distributions instead of in-tree validation to perform the regression checking. This would have the advantage of also testing the substantial build system infrastructure that creates distribution tarballs. However, due to a number of problems (tickets #14392, #14411 & #14412) this is currently not possible. We still want to eventually move to this more comprehensive scheme, but for now we are moving this of the critical path to get the initial system running. > > Mateusz and I contacted AppVeyor concerning a more generous than the standard limit for OSS projects (for the Windows build), but haven’t heard back yet. > > In addition to merging the work so far (#14416), the next steps are > > * support for PR builds and integration with Phabricator and > * Windows support. > Thanks for the update Manuel! Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From manuel.chakravarty at tweag.io Fri Nov 10 02:32:07 2017 From: manuel.chakravarty at tweag.io (Manuel M T Chakravarty) Date: Fri, 10 Nov 2017 13:32:07 +1100 Subject: [GHC DevOps Group] New release process In-Reply-To: References: Message-ID: As Ben has just announced on ghc-dev, we are getting close to cutting the 8.4 branch, which marks the start of working towards the new release process: https://mail.haskell.org/pipermail/ghc-devs/2017-November/015038.html In the meantime, I have summarised our plan as well as the discussion that we had on this thread so far on a new Trac Wiki page: https://mail.haskell.org/pipermail/ghc-devs/2017-November/015038.html I will also keep track of how the process unfolds on this page, so you can all easily get up to speed by just checking that page. The previous discussion on this thread ebbed off, but that doesn’t mean that all concerns have been alleviated. Moreover, the interaction with the 3 versions library policy is still unclear and we should talk with the Core Libraries Committee. What do you all think? Ben, can you please double check what I wrote to make sure I didn’t miss anything or get anything wrong? Cheers, Manuel -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben at well-typed.com Fri Nov 10 19:51:24 2017 From: ben at well-typed.com (Ben Gamari) Date: Fri, 10 Nov 2017 14:51:24 -0500 Subject: [GHC DevOps Group] New release process In-Reply-To: References: Message-ID: <878tfd7w09.fsf@ben-laptop.smart-cactus.org> Manuel M T Chakravarty writes: > As Ben has just announced on ghc-dev, we are getting close to cutting the 8.4 branch, which marks the start of working towards the new release process: > > https://mail.haskell.org/pipermail/ghc-devs/2017-November/015038.html > > In the meantime, I have summarised our plan as well as the discussion that we had on this thread so far on a new Trac Wiki page: > > https://mail.haskell.org/pipermail/ghc-devs/2017-November/015038.html > It seems there was some clipboard mishap here :) I think this second link was supposed to be https://ghc.haskell.org/trac/ghc/wiki/WorkingConventions/Releases/NewSchedule Thanks for writing this down, Manuel. Yes, this looks like it reflects what we discussed quite well. I agree that bringing up the change with the CLC is a necessary step. Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From ben at well-typed.com Sun Nov 12 01:39:40 2017 From: ben at well-typed.com (Ben Gamari) Date: Sat, 11 Nov 2017 20:39:40 -0500 Subject: [GHC DevOps Group] CircleCI job accounting question Message-ID: <8760agnulr.fsf@ben-laptop.smart-cactus.org> Hi Manuel, I seem to recall you earlier mentioning that CircleCI jobs associated with forks of a repository do not count towards the job limit of the forked repository. How certain are you that this is the case? I ask because I have done a bit of work on the CircleCI infrastructure which I pushed to my GitHub fork earlier today. However, the build [1] has inexplicably still not been started. Given that the ghc/ghc repository is still catching up on its own builds, I suspect that both repositories are drawing against the same resource limit. Unfortunately I have been unable to find compelling evidence one way or the other in the CircleCI documentation. If it is true that builds of forks count against the forked repository's resource limit then we will need to make the appropriate accomodations in our resource planning. In other news, I have opened another support ticket about the hanging Darwin build issue, which still appears to be occurring. I'll update you and Mateusz when I hear back. Thanks again for everything you and Mateusz have done on this. Cheers, - Ben [1] https://circleci.com/gh/bgamari/ghc/15 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From jonas.chevalier at tweag.io Sun Nov 12 13:09:49 2017 From: jonas.chevalier at tweag.io (Jonas Chevalier) Date: Sun, 12 Nov 2017 13:09:49 +0000 Subject: [GHC DevOps Group] CircleCI job accounting question In-Reply-To: <8760agnulr.fsf@ben-laptop.smart-cactus.org> References: <8760agnulr.fsf@ben-laptop.smart-cactus.org> Message-ID: Hi Ben, I'm interjecting since I have been helping setting things up in the background. Send an email to support at circleci.zendesk.com and mention that you would like to enable the support for "macOS CircleCI 2.0" and "Configurable resource class" on your repository, which is a fork of ghc/ghc. In general it takes 1 business day to get it activated. Cheers, Jonas On Sun, Nov 12, 2017 at 1:39 AM Ben Gamari wrote: > Hi Manuel, > > I seem to recall you earlier mentioning that CircleCI jobs associated > with forks of a repository do not count towards the job limit of the > forked repository. How certain are you that this is the case? > > I ask because I have done a bit of work on the CircleCI infrastructure > which I pushed to my GitHub fork earlier today. However, the build [1] > has inexplicably still not been started. Given that the ghc/ghc > repository is still catching up on its own builds, I suspect that both > repositories are drawing against the same resource limit. Unfortunately > I have been unable to find compelling evidence one way or the other in > the CircleCI documentation. > > If it is true that builds of forks count against the forked repository's > resource limit then we will need to make the appropriate accomodations > in our resource planning. > > In other news, I have opened another support ticket about the hanging > Darwin > build issue, which still appears to be occurring. I'll update you and > Mateusz when I hear back. > > Thanks again for everything you and Mateusz have done on this. > > Cheers, > > - Ben > > > [1] https://circleci.com/gh/bgamari/ghc/15 > _______________________________________________ > Ghc-devops-group mailing list > Ghc-devops-group at haskell.org > https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devops-group > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben at well-typed.com Sun Nov 12 16:37:24 2017 From: ben at well-typed.com (Ben Gamari) Date: Sun, 12 Nov 2017 11:37:24 -0500 Subject: [GHC DevOps Group] CircleCI job accounting question In-Reply-To: References: <8760agnulr.fsf@ben-laptop.smart-cactus.org> Message-ID: <87zi7rmp1c.fsf@ben-laptop.smart-cactus.org> Jonas Chevalier writes: > Hi Ben, > > I'm interjecting since I have been helping setting things up in the > background. > Thanks Jonas! I'll be sure to CC you in the future. > Send an email to support at circleci.zendesk.com and mention that you would > like to enable the support for "macOS CircleCI 2.0" and "Configurable > resource class" on your repository, which is a fork of ghc/ghc. In general > it takes 1 business day to get it activated. > Thanks. Will do. However, I'm a bit confused; does configurable resource limit support have any bearing on whether my fork's builds count against ghc/ghc's resource limit? My understanding is that it does not; rather, it merely allows the CircleCI configuration to specify that it wants to run on a large machine using the `resource_class` key in `.circleci.yml`. Is this correct? Will all users who test using their fork need to ask for this to be enabled? If so, is what I said in my original message correct? Should we expect all forks of ghc/ghc to count against a single resource limit? Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From manuel.chakravarty at tweag.io Mon Nov 13 02:22:25 2017 From: manuel.chakravarty at tweag.io (Manuel M T Chakravarty) Date: Mon, 13 Nov 2017 13:22:25 +1100 Subject: [GHC DevOps Group] CircleCI job accounting question In-Reply-To: <8760agnulr.fsf@ben-laptop.smart-cactus.org> References: <8760agnulr.fsf@ben-laptop.smart-cactus.org> Message-ID: <0B8947A7-CF4E-4803-828E-2DC1FBC6F4F0@tweag.io> Hi Ben, Firstly re https://github.com/ghc/ghc/commit/5f158bc1a7eedf8680f4a1e612ca34daa05e0029 CircleCI seems to now use an 8CPU (xlarge) instance to run the GHC builds. This is after Jonas asked them to activate it. You can see that, e.g., by looking at the ”Resource” value on the last few > builds. Secondly, I am not sure what is happening with your fork, but forks cannot all count against the main repo. For example, Tweag’s GHC fork has been running for a long time independently and also has already been using the xlarge instances (which weren’t even enabled for the main GHC repo). However, re your question to Jonas, > Will all users who test using their fork need to ask for this > to be enabled? Yes, my understanding is that individual GitHub accounts forking GHC who want the additional resources (for xlarge and for macOS) will have to individually request the additional resources from CircleCI. That is the ”price” for it not to be charged against the same account. (We need to document that.) Re Darwin, isn’t that a different issue now? I just looked at the last Darwin build ”ghc / ghc / master #272” and the build seems to be fine, but it chokes on uploading the build artefacts. Mateusz, what do you think? Cheers, Manuel PS: If one of the Tweag’ers could be added to the GitHub GHC org, we could take care of things like activating xlarge more easily… > Ben Gamari : > > Hi Manuel, > > I seem to recall you earlier mentioning that CircleCI jobs associated > with forks of a repository do not count towards the job limit of the > forked repository. How certain are you that this is the case? > > I ask because I have done a bit of work on the CircleCI infrastructure > which I pushed to my GitHub fork earlier today. However, the build [1] > has inexplicably still not been started. Given that the ghc/ghc > repository is still catching up on its own builds, I suspect that both > repositories are drawing against the same resource limit. Unfortunately > I have been unable to find compelling evidence one way or the other in > the CircleCI documentation. > > If it is true that builds of forks count against the forked repository's > resource limit then we will need to make the appropriate accomodations > in our resource planning. > > In other news, I have opened another support ticket about the hanging Darwin > build issue, which still appears to be occurring. I'll update you and > Mateusz when I hear back. > > Thanks again for everything you and Mateusz have done on this. > > Cheers, > > - Ben > > > [1] https://circleci.com/gh/bgamari/ghc/15 -------------- next part -------------- An HTML attachment was scrubbed... URL: From manuel.chakravarty at tweag.io Mon Nov 13 02:35:50 2017 From: manuel.chakravarty at tweag.io (Manuel M T Chakravarty) Date: Mon, 13 Nov 2017 13:35:50 +1100 Subject: [GHC DevOps Group] New release process In-Reply-To: <878tfd7w09.fsf@ben-laptop.smart-cactus.org> References: <878tfd7w09.fsf@ben-laptop.smart-cactus.org> Message-ID: <87732D94-0BB7-4E54-A247-B1B982957557@tweag.io> Oops, yes, clipboard finger twist :0 And, yes, it should have been the link you cite. Concerning the CLC, who should approach them? I’d be happy to do it, but don’t want to step on anybody’s feet. Cheers Manuel > Am 11.11.2017 um 06:51 schrieb Ben Gamari : > > Manuel M T Chakravarty writes: > >> As Ben has just announced on ghc-dev, we are getting close to cutting the 8.4 branch, which marks the start of working towards the new release process: >> >> https://mail.haskell.org/pipermail/ghc-devs/2017-November/015038.html >> >> In the meantime, I have summarised our plan as well as the discussion that we had on this thread so far on a new Trac Wiki page: >> >> https://mail.haskell.org/pipermail/ghc-devs/2017-November/015038.html >> > It seems there was some clipboard mishap here :) > > I think this second link was supposed to be > > https://ghc.haskell.org/trac/ghc/wiki/WorkingConventions/Releases/NewSchedule > > Thanks for writing this down, Manuel. Yes, this looks like it reflects > what we discussed quite well. > > I agree that bringing up the change with the CLC is a necessary step. > > Cheers, > > - Ben From ben at well-typed.com Mon Nov 13 03:23:34 2017 From: ben at well-typed.com (Ben Gamari) Date: Sun, 12 Nov 2017 22:23:34 -0500 Subject: [GHC DevOps Group] CircleCI job accounting question In-Reply-To: <0B8947A7-CF4E-4803-828E-2DC1FBC6F4F0@tweag.io> References: <8760agnulr.fsf@ben-laptop.smart-cactus.org> <0B8947A7-CF4E-4803-828E-2DC1FBC6F4F0@tweag.io> Message-ID: <8760aen9ov.fsf@ben-laptop.smart-cactus.org> Manuel M T Chakravarty writes: > Hi Ben, > > Firstly re > > https://github.com/ghc/ghc/commit/5f158bc1a7eedf8680f4a1e612ca34daa05e0029 > > CircleCI seems to now use an 8CPU (xlarge) instance to run the GHC > builds. This is after Jonas asked them to activate it. You can see > that, e.g., by looking at the ”Resource” value on the last few > > builds. > Really we ought to just determine the CPU count dynamically. This is what the validate script does. Unfortunately it seems the CircleCI does not provide an environment variable with this information and variables defined the YAML environment section aren't interpolated, so doing so will be a bit messy. > Secondly, I am not sure what is happening with your fork, but forks > cannot all count against the main repo. For example, Tweag’s GHC fork > has been running for a long time independently and also has already > been using the xlarge instances (which weren’t even enabled for the > main GHC repo). > Very odd. I'll admit I'm quite flummoxed. > However, re your question to Jonas, > >> Will all users who test using their fork need to ask for this >> to be enabled? > > Yes, my understanding is that individual GitHub accounts forking GHC > who want the additional resources (for xlarge and for macOS) will have > to individually request the additional resources from CircleCI. That > is the ”price” for it not to be charged against the same account. (We > need to document that.) > > Re Darwin, isn’t that a different issue now? I just looked at the last > Darwin build ”ghc / ghc / master #272” and the build seems to be fine, > but it chokes on uploading the build artefacts. > Yes, this is a different issue from the one we were seeing earlier. I believe Mateusz has also opened a ticket about it. > Mateusz, what do you think? > > Cheers, > Manuel > > PS: If one of the Tweag’ers could be added to the GitHub GHC org, we > could take care of things like activating xlarge more easily… > You should find an invitation in your inbox. Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From manuel.chakravarty at tweag.io Wed Nov 15 00:09:14 2017 From: manuel.chakravarty at tweag.io (Manuel M T Chakravarty) Date: Wed, 15 Nov 2017 11:09:14 +1100 Subject: [GHC DevOps Group] CircleCI job accounting question In-Reply-To: <8760aen9ov.fsf@ben-laptop.smart-cactus.org> References: <8760agnulr.fsf@ben-laptop.smart-cactus.org> <0B8947A7-CF4E-4803-828E-2DC1FBC6F4F0@tweag.io> <8760aen9ov.fsf@ben-laptop.smart-cactus.org> Message-ID: <54496A02-05EE-43EF-B717-612829628D90@tweag.io> > Ben Gamari : > > Manuel M T Chakravarty writes: > >> Hi Ben, >> >> Firstly re >> >> https://github.com/ghc/ghc/commit/5f158bc1a7eedf8680f4a1e612ca34daa05e0029 >> >> CircleCI seems to now use an 8CPU (xlarge) instance to run the GHC >> builds. This is after Jonas asked them to activate it. You can see >> that, e.g., by looking at the ”Resource” value on the last few >> > builds. >> > Really we ought to just determine the CPU count dynamically. This is > what the validate script does. Unfortunately it seems the CircleCI does > not provide an environment variable with this information and variables > defined the YAML environment section aren't interpolated, so doing so > will be a bit messy. Yes, is annoying. On the other hand, you’d think, we are not the only people having this problem. >> However, re your question to Jonas, >> >>> Will all users who test using their fork need to ask for this >>> to be enabled? >> >> Yes, my understanding is that individual GitHub accounts forking GHC >> who want the additional resources (for xlarge and for macOS) will have >> to individually request the additional resources from CircleCI. That >> is the ”price” for it not to be charged against the same account. (We >> need to document that.) >> >> Re Darwin, isn’t that a different issue now? I just looked at the last >> Darwin build ”ghc / ghc / master #272” and the build seems to be fine, >> but it chokes on uploading the build artefacts. >> > Yes, this is a different issue from the one we were seeing earlier. I > believe Mateusz has also opened a ticket about it. I am just looking at the CircleCI page while ”ghc / ghc / master #289” as it is running. It all works nicely, but now it is somehow hanging in the ”Uploading artefacts” step for already more than 22min. Interestingly, it does report Uploading /tmp/artifacts to tmp/artifacts Uploaded /tmp/artifacts/ghc-8.3.20171114-x86_64-apple-darwin.tar.xz which suggests that it is actually finished with uploading. It just doesn’t realise that it is done… >> Mateusz, what do you think? >> >> Cheers, >> Manuel >> >> PS: If one of the Tweag’ers could be added to the GitHub GHC org, we >> could take care of things like activating xlarge more easily… >> > You should find an invitation in your inbox. Thank you! Cheers, Manuel -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben at well-typed.com Wed Nov 15 20:42:40 2017 From: ben at well-typed.com (Ben Gamari) Date: Wed, 15 Nov 2017 15:42:40 -0500 Subject: [GHC DevOps Group] CircleCI job accounting question In-Reply-To: <54496A02-05EE-43EF-B717-612829628D90@tweag.io> References: <8760agnulr.fsf@ben-laptop.smart-cactus.org> <0B8947A7-CF4E-4803-828E-2DC1FBC6F4F0@tweag.io> <8760aen9ov.fsf@ben-laptop.smart-cactus.org> <54496A02-05EE-43EF-B717-612829628D90@tweag.io> Message-ID: <87zi7njmth.fsf@ben-laptop.smart-cactus.org> Manuel M T Chakravarty writes: >> Ben Gamari : >> >> Really we ought to just determine the CPU count dynamically. This is >> what the validate script does. Unfortunately it seems the CircleCI does >> not provide an environment variable with this information and variables >> defined the YAML environment section aren't interpolated, so doing so >> will be a bit messy. > > Yes, is annoying. On the other hand, you’d think, we are not the only people having this problem. > Indeed we aren't; for instance, see [1] and [2]. [1] https://discuss.circleci.com/t/how-to-add-a-path-to-path-in-circle-2-0/11554 [2] https://discuss.circleci.com/t/circle-ci-v2-and-android-memory-issues/11207/9 >> Yes, this is a different issue from the one we were seeing earlier. I >> believe Mateusz has also opened a ticket about it. > > I am just looking at the CircleCI page while ”ghc / ghc / master #289” as it is running. It all works nicely, but now it is somehow hanging in the ”Uploading artefacts” step for already more than 22min. Interestingly, it does report > > Uploading /tmp/artifacts to tmp/artifacts > Uploaded /tmp/artifacts/ghc-8.3.20171114-x86_64-apple-darwin.tar.xz > > which suggests that it is actually finished with uploading. It just doesn’t realise that it is done… > Indeed. CircleCI support replied to my request this morning. Unfortunately they haven't yet offered a solution. I'll continue to update you all as the ticket advances. On another note, what is the status of Appveyor? Has anyone started on this? If not I may try bringing over Hadrian's Appveyor configuration. Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From manuel.chakravarty at tweag.io Wed Nov 15 23:01:23 2017 From: manuel.chakravarty at tweag.io (Manuel M T Chakravarty) Date: Thu, 16 Nov 2017 10:01:23 +1100 Subject: [GHC DevOps Group] CircleCI job accounting question In-Reply-To: <87zi7njmth.fsf@ben-laptop.smart-cactus.org> References: <8760agnulr.fsf@ben-laptop.smart-cactus.org> <0B8947A7-CF4E-4803-828E-2DC1FBC6F4F0@tweag.io> <8760aen9ov.fsf@ben-laptop.smart-cactus.org> <54496A02-05EE-43EF-B717-612829628D90@tweag.io> <87zi7njmth.fsf@ben-laptop.smart-cactus.org> Message-ID: > Ben Gamari : > > Manuel M T Chakravarty writes: > >>> Ben Gamari : >>> >>> Really we ought to just determine the CPU count dynamically. This is >>> what the validate script does. Unfortunately it seems the CircleCI does >>> not provide an environment variable with this information and variables >>> defined the YAML environment section aren't interpolated, so doing so >>> will be a bit messy. >> >> Yes, is annoying. On the other hand, you’d think, we are not the only people having this problem. >> > Indeed we aren't; for instance, see [1] and [2]. > > [1] https://discuss.circleci.com/t/how-to-add-a-path-to-path-in-circle-2-0/11554 > [2] https://discuss.circleci.com/t/circle-ci-v2-and-android-memory-issues/11207/9 Judging by the comments on the forum, including https://discuss.circleci.com/t/environment-variable-interpolation-throughout-circleci-config-yml/16300 expanding environment variables in config files is planned for CircleCI 2.0, but not yet implemented. Hence, we could just punt on this for now and document that people who for the CircleCI config need to ask to get xlarge enabled (unless they have a paid account anyway). >>> Yes, this is a different issue from the one we were seeing earlier. I >>> believe Mateusz has also opened a ticket about it. >> >> I am just looking at the CircleCI page while ”ghc / ghc / master #289” as it is running. It all works nicely, but now it is somehow hanging in the ”Uploading artefacts” step for already more than 22min. Interestingly, it does report >> >> Uploading /tmp/artifacts to tmp/artifacts >> Uploaded /tmp/artifacts/ghc-8.3.20171114-x86_64-apple-darwin.tar.xz >> >> which suggests that it is actually finished with uploading. It just doesn’t realise that it is done… >> > Indeed. CircleCI support replied to my request this morning. > Unfortunately they haven't yet offered a solution. I'll continue to > update you all as the ticket advances. What did they reply? Just acknowledged the problem? > On another note, what is the status of Appveyor? Has anyone started on > this? If not I may try bringing over Hadrian’s Appveyor configuration. Mateusz had a first stab https://github.com/tweag/ghc/blob/tweag/circleci-macos/appveyor.yml but got stuck in the default resource limits. We emailed them with a request, but there was no answer so far. I’ll follow up on it. What does Hadrian do? Do they use default resource limits? Cheers, Manuel -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben at well-typed.com Thu Nov 16 00:23:22 2017 From: ben at well-typed.com (Ben Gamari) Date: Wed, 15 Nov 2017 19:23:22 -0500 Subject: [GHC DevOps Group] CircleCI job accounting question In-Reply-To: References: <8760agnulr.fsf@ben-laptop.smart-cactus.org> <0B8947A7-CF4E-4803-828E-2DC1FBC6F4F0@tweag.io> <8760aen9ov.fsf@ben-laptop.smart-cactus.org> <54496A02-05EE-43EF-B717-612829628D90@tweag.io> <87zi7njmth.fsf@ben-laptop.smart-cactus.org> Message-ID: <87po8jjclk.fsf@ben-laptop.smart-cactus.org> Manuel M T Chakravarty writes: >> Ben Gamari : >> >> Manuel M T Chakravarty writes: >> >>> >>> Yes, is annoying. On the other hand, you’d think, we are not the only people having this problem. >>> >> Indeed we aren't; for instance, see [1] and [2]. >> >> [1] https://discuss.circleci.com/t/how-to-add-a-path-to-path-in-circle-2-0/11554 >> [2] https://discuss.circleci.com/t/circle-ci-v2-and-android-memory-issues/11207/9 > > Judging by the comments on the forum, including > > https://discuss.circleci.com/t/environment-variable-interpolation-throughout-circleci-config-yml/16300 > > expanding environment variables in config files is planned for > CircleCI 2.0, but not yet implemented. Hence, we could just punt on > this for now and document that people who for the CircleCI config need > to ask to get xlarge enabled (unless they have a paid account anyway). > Yes, I'm not considering this a high priority issue. I've opened #14470 to track it. >> Indeed. CircleCI support replied to my request this morning. >> Unfortunately they haven't yet offered a solution. I'll continue to >> update you all as the ticket advances. > > What did they reply? Just acknowledged the problem? > They claimed that the build had failed citing the log. Specifically he said, > Your request (26406) has been updated. To add additional comments, reply to this email. > ---------------------------------------------- > > Joseph Becher, Nov 15, 07:52 PST > > Hi Ben, > > I'm not seeing that it gets to the artifact upload step. It looks like > the build is failing with a number of missing files, you can view the > full log for that build here > https://circleci.com/api/v1.1/project/github/ghc/ghc/269/output/106/0 I can only imagine that he saw the Haddock warnings and concluded that they were fatal. I replied asking for clarification. >> On another note, what is the status of Appveyor? Has anyone started on >> this? If not I may try bringing over Hadrian’s Appveyor configuration. > > Mateusz had a first stab > Great! > https://github.com/tweag/ghc/blob/tweag/circleci-macos/appveyor.yml > > but got stuck in the default resource limits. We emailed them with a > request, but there was no answer so far. I’ll follow up on it. > > What does Hadrian do? Do they use default resource limits? > That is a good question. I've asked him. I'll let you know when I hear back. Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From ben at well-typed.com Thu Nov 16 04:12:37 2017 From: ben at well-typed.com (Ben Gamari) Date: Wed, 15 Nov 2017 23:12:37 -0500 Subject: [GHC DevOps Group] CircleCI job accounting question In-Reply-To: <87po8jjclk.fsf@ben-laptop.smart-cactus.org> References: <8760agnulr.fsf@ben-laptop.smart-cactus.org> <0B8947A7-CF4E-4803-828E-2DC1FBC6F4F0@tweag.io> <8760aen9ov.fsf@ben-laptop.smart-cactus.org> <54496A02-05EE-43EF-B717-612829628D90@tweag.io> <87zi7njmth.fsf@ben-laptop.smart-cactus.org> <87po8jjclk.fsf@ben-laptop.smart-cactus.org> Message-ID: <87ineakgk2.fsf@ben-laptop.smart-cactus.org> Ben Gamari writes: > Manuel M T Chakravarty writes: >> >> but got stuck in the default resource limits. We emailed them with a >> request, but there was no answer so far. I’ll follow up on it. >> >> What does Hadrian do? Do they use default resource limits? >> > That is a good question. I've asked him. I'll let you know when I hear > back. Indeed Andrey says that they are using the default resource limits (1 hour execution time). That being said, it looks like they barely fit (with most builds taking around 50 minutes) and they are using the `quickest` build flavor without running the testsuite. I just checked and on my four-core VM a full Windows validate takes around 1.75 hours. It seems like Appveyor typically [1] offers open-source projects up to 1.5 hours upon request. That being said, Rust has apparently worked out something as their builds [2] are typically a bit longer than 2 hours. I know for a fact that they pay for Travis, so they likely pay for Appveyor as well. It appears that Appveyor runs builds [3] on 2 vCPU instances so we might need to scale my four-core time measurement up a bit (although almost certainly not by a factor of two) Cheers, - Ben [1] https://github.com/appveyor/ci/issues/517 [2] https://ci.appveyor.com/project/rust-lang/rust/history [3] https://www.appveyor.com/docs/build-environment/#build-vm-configurations -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From ben at well-typed.com Wed Nov 22 03:08:51 2017 From: ben at well-typed.com (Ben Gamari) Date: Tue, 21 Nov 2017 22:08:51 -0500 Subject: [GHC DevOps Group] CircleCI job accounting question In-Reply-To: References: <8760agnulr.fsf@ben-laptop.smart-cactus.org> <0B8947A7-CF4E-4803-828E-2DC1FBC6F4F0@tweag.io> <8760aen9ov.fsf@ben-laptop.smart-cactus.org> <54496A02-05EE-43EF-B717-612829628D90@tweag.io> <87zi7njmth.fsf@ben-laptop.smart-cactus.org> Message-ID: <87mv3f80xv.fsf@ben-laptop.smart-cactus.org> Manuel M T Chakravarty writes: > Mateusz had a first stab > > https://github.com/tweag/ghc/blob/tweag/circleci-macos/appveyor.yml > > but got stuck in the default resource limits. We emailed them with a > request, but there was no answer so far. I’ll follow up on it. > Any update on this? For the record, I have confirmed with the Rustaceans that Mozilla indeed pays for their usage. In other news, * I have disabled artifact collection in the OS X build for now. * It appears that CircleCI only builds the head commits of pushes. Making this configurable has been a feature request for nearly a year now, so it looks like we will need to work around this. I briefly looked into setting up some automation to trigger builds on otherwise untested commits, but ran into apparent API bugginess. It looks like we'll just need to ensure that contributors push at most one commit at a time for now to ensure all commits get testing. See GHC #14505 for details. * I have tried enabling testing of Harbormaster Differentials via CircleCI. Unfortunately it appears that CircleCI only supports testing repositories hosted on GitHub. There are a few ways in which we could proceed, a. Move ghc's staging area (the repository where Arcanist pushes patches submitted with `arc diff`) to GitHub. This, however, would require that we manually manage push privileges to this repository. b. Try to work around the issue by mirroring GHC's staging area to GitHub and manually trigger CircleCI builds. * I have been honing the Hadrian test infrastructure; I'm currently waiting on a build, but I expect this attempt will pass, at which point I will merge it. Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From manuel.chakravarty at tweag.io Wed Nov 22 14:02:32 2017 From: manuel.chakravarty at tweag.io (Manuel M T Chakravarty) Date: Wed, 22 Nov 2017 15:02:32 +0100 Subject: [GHC DevOps Group] CircleCI job accounting question In-Reply-To: <87mv3f80xv.fsf@ben-laptop.smart-cactus.org> References: <8760agnulr.fsf@ben-laptop.smart-cactus.org> <0B8947A7-CF4E-4803-828E-2DC1FBC6F4F0@tweag.io> <8760aen9ov.fsf@ben-laptop.smart-cactus.org> <54496A02-05EE-43EF-B717-612829628D90@tweag.io> <87zi7njmth.fsf@ben-laptop.smart-cactus.org> <87mv3f80xv.fsf@ben-laptop.smart-cactus.org> Message-ID: > Ben Gamari : > > Manuel M T Chakravarty writes: > >> Mateusz had a first stab >> >> https://github.com/tweag/ghc/blob/tweag/circleci-macos/appveyor.yml >> >> but got stuck in the default resource limits. We emailed them with a >> request, but there was no answer so far. I’ll follow up on it. >> > Any update on this? For the record, I have confirmed with the Rustaceans > that Mozilla indeed pays for their usage. No, sorry, I have been completely taken out with travelling and conference for the last week. (Just arrived in the Netherlands.) > In other news, > > * I have disabled artifact collection in the OS X build for now. Ok. (From the ticket, I saw that the CircleCI people seem to be actively looking into this.) > * It appears that CircleCI only builds the head commits of pushes. > Making this configurable has been a feature request for nearly a year > now, so it looks like we will need to work around this. I briefly > looked into setting up some automation to trigger builds on otherwise > untested commits, but ran into apparent API bugginess. It looks like > we'll just need to ensure that contributors push at most one commit > at a time for now to ensure all commits get testing. See GHC #14505 > for details. Why do we need the intermediate builds exactly? Wouldn’t they usually fail? (When I do PRs with multiple commits, the state of the tree between this commits will usually not be well-defined.) > * I have tried enabling testing of Harbormaster Differentials via > CircleCI. Unfortunately it appears that CircleCI only supports > testing repositories hosted on GitHub. There are a few ways in which > we could proceed, > > a. Move ghc's staging area (the repository where Arcanist pushes > patches submitted with `arc diff`) to GitHub. This, however, would > require that we manually manage push privileges to this repository. What do you mean by manually manage push privileges? In what way is that not manually at the moment? > b. Try to work around the issue by mirroring GHC's staging area to > GitHub and manually trigger CircleCI builds. Is the manual triggering necessary, because Harbormaster would need to wait until the repo is triggered (which it can’t)? > * I have been honing the Hadrian test infrastructure; I'm currently > waiting on a build, but I expect this attempt will pass, at which point > I will merge it. Great! Cheers, Manuel From ben at well-typed.com Wed Nov 22 15:31:24 2017 From: ben at well-typed.com (Ben Gamari) Date: Wed, 22 Nov 2017 10:31:24 -0500 Subject: [GHC DevOps Group] CircleCI job accounting question In-Reply-To: References: <8760agnulr.fsf@ben-laptop.smart-cactus.org> <0B8947A7-CF4E-4803-828E-2DC1FBC6F4F0@tweag.io> <8760aen9ov.fsf@ben-laptop.smart-cactus.org> <54496A02-05EE-43EF-B717-612829628D90@tweag.io> <87zi7njmth.fsf@ben-laptop.smart-cactus.org> <87mv3f80xv.fsf@ben-laptop.smart-cactus.org> Message-ID: <87d14a8h4s.fsf@ben-laptop.smart-cactus.org> Manuel M T Chakravarty writes: >> Ben Gamari : >> >> Manuel M T Chakravarty writes: >> >>> Mateusz had a first stab >>> >>> https://github.com/tweag/ghc/blob/tweag/circleci-macos/appveyor.yml >>> >>> but got stuck in the default resource limits. We emailed them with a >>> request, but there was no answer so far. I’ll follow up on it. >>> >> Any update on this? For the record, I have confirmed with the Rustaceans >> that Mozilla indeed pays for their usage. > > No, sorry, I have been completely taken out with travelling and > conference for the last week. (Just arrived in the Netherlands.) Quite alright; just wondering. >> * It appears that CircleCI only builds the head commits of pushes. >> Making this configurable has been a feature request for nearly a year >> now, so it looks like we will need to work around this. I briefly >> looked into setting up some automation to trigger builds on otherwise >> untested commits, but ran into apparent API bugginess. It looks like >> we'll just need to ensure that contributors push at most one commit >> at a time for now to ensure all commits get testing. See GHC #14505 >> for details. > > Why do we need the intermediate builds exactly? Wouldn’t they usually > fail? (When I do PRs with multiple commits, the state of the tree > between this commits will usually not be well-defined.) No, every commit should build. This is in part a difference between Phabricator's patch-based model and GitHub's feature branch model. However, many projects using the latter also demand that all intermediate commits must be atomic, buildable changes. Sacrificing this property greatly complicates bisection. Building all intermediates is desireable as ultimately we would like to preserve per-commit build artifacts for the last few months of commits to enable easy bisection. >> * I have tried enabling testing of Harbormaster Differentials via >> CircleCI. Unfortunately it appears that CircleCI only supports >> testing repositories hosted on GitHub. There are a few ways in which >> we could proceed, >> >> a. Move ghc's staging area (the repository where Arcanist pushes >> patches submitted with `arc diff`) to GitHub. This, however, would >> require that we manually manage push privileges to this repository. > > What do you mean by manually manage push privileges? In what way is > that not manually at the moment? As long as a user had added a key to their Phabricator account pushing to the staging area will "just work". It requires no intervention from me. I believe in the event that we moved to GitHub I would need to manually grant users commit privileges to the staging area. >> b. Try to work around the issue by mirroring GHC's staging area to >> GitHub and manually trigger CircleCI builds. > > Is the manual triggering necessary, because Harbormaster would need to > wait until the repo is triggered (which it can’t)? In general this whole mirroring situation doesn't appear to be a use-case that Phabricator's CircleCI integration supports. It demands that the staging area of the tested repository is hosted on GitHub. >> * I have been honing the Hadrian test infrastructure; I'm currently >> waiting on a build, but I expect this attempt will pass, at which point >> I will merge it. > > Great! Sadly the build appears to reliably hang with no output. I'll need to look into this. Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From marlowsd at gmail.com Wed Nov 22 17:09:46 2017 From: marlowsd at gmail.com (Simon Marlow) Date: Wed, 22 Nov 2017 17:09:46 +0000 Subject: [GHC DevOps Group] CircleCI job accounting question In-Reply-To: <87d14a8h4s.fsf@ben-laptop.smart-cactus.org> References: <8760agnulr.fsf@ben-laptop.smart-cactus.org> <0B8947A7-CF4E-4803-828E-2DC1FBC6F4F0@tweag.io> <8760aen9ov.fsf@ben-laptop.smart-cactus.org> <54496A02-05EE-43EF-B717-612829628D90@tweag.io> <87zi7njmth.fsf@ben-laptop.smart-cactus.org> <87mv3f80xv.fsf@ben-laptop.smart-cactus.org> <87d14a8h4s.fsf@ben-laptop.smart-cactus.org> Message-ID: On 22 November 2017 at 15:31, Ben Gamari wrote: > Manuel M T Chakravarty writes: > > Why do we need the intermediate builds exactly? Wouldn’t they usually > > fail? (When I do PRs with multiple commits, the state of the tree > > between this commits will usually not be well-defined.) > > No, every commit should build. This is in part a difference between > Phabricator's patch-based model and GitHub's feature branch model. > However, many projects using the latter also demand that all > intermediate commits must be atomic, buildable changes. Sacrificing this > property greatly complicates bisection. > > Building all intermediates is desireable as ultimately we would like to > preserve per-commit build artifacts for the last few months of commits > to enable easy bisection. > I don't quite understand this. Yes building all commits is desirable, but in the case of Phabricator each revision is going to be a single commit, no? So why is this an issue? Or is it an issue only for github PRs? I thought we had decided (somewhere, I forget where) that we were going to require PRs to be squashed before pushing? Cheers Simon > > >> * I have tried enabling testing of Harbormaster Differentials via > >> CircleCI. Unfortunately it appears that CircleCI only supports > >> testing repositories hosted on GitHub. There are a few ways in which > >> we could proceed, > >> > >> a. Move ghc's staging area (the repository where Arcanist pushes > >> patches submitted with `arc diff`) to GitHub. This, however, would > >> require that we manually manage push privileges to this > repository. > > > > What do you mean by manually manage push privileges? In what way is > > that not manually at the moment? > > As long as a user had added a key to their Phabricator account pushing > to the staging area will "just work". It requires no intervention from > me. I believe in the event that we moved to GitHub I would need to > manually grant users commit privileges to the staging area. > > >> b. Try to work around the issue by mirroring GHC's staging area to > >> GitHub and manually trigger CircleCI builds. > > > > Is the manual triggering necessary, because Harbormaster would need to > > wait until the repo is triggered (which it can’t)? > > In general this whole mirroring situation doesn't appear to be a > use-case that Phabricator's CircleCI integration supports. It demands > that the staging area of the tested repository is hosted on GitHub. > > >> * I have been honing the Hadrian test infrastructure; I'm currently > >> waiting on a build, but I expect this attempt will pass, at which > point > >> I will merge it. > > > > Great! > > Sadly the build appears to reliably hang with no output. I'll need to > look into this. > > Cheers, > > - Ben > > _______________________________________________ > Ghc-devops-group mailing list > Ghc-devops-group at haskell.org > https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devops-group > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben at well-typed.com Wed Nov 22 19:15:27 2017 From: ben at well-typed.com (Ben Gamari) Date: Wed, 22 Nov 2017 14:15:27 -0500 Subject: [GHC DevOps Group] CircleCI job accounting question In-Reply-To: References: <8760agnulr.fsf@ben-laptop.smart-cactus.org> <0B8947A7-CF4E-4803-828E-2DC1FBC6F4F0@tweag.io> <8760aen9ov.fsf@ben-laptop.smart-cactus.org> <54496A02-05EE-43EF-B717-612829628D90@tweag.io> <87zi7njmth.fsf@ben-laptop.smart-cactus.org> <87mv3f80xv.fsf@ben-laptop.smart-cactus.org> <87d14a8h4s.fsf@ben-laptop.smart-cactus.org> Message-ID: <874lpm86ra.fsf@ben-laptop.smart-cactus.org> Simon Marlow writes: > On 22 November 2017 at 15:31, Ben Gamari wrote: > >> Manuel M T Chakravarty writes: >> > Why do we need the intermediate builds exactly? Wouldn’t they usually >> > fail? (When I do PRs with multiple commits, the state of the tree >> > between this commits will usually not be well-defined.) >> >> No, every commit should build. This is in part a difference between >> Phabricator's patch-based model and GitHub's feature branch model. >> However, many projects using the latter also demand that all >> intermediate commits must be atomic, buildable changes. Sacrificing this >> property greatly complicates bisection. >> >> Building all intermediates is desireable as ultimately we would like to >> preserve per-commit build artifacts for the last few months of commits >> to enable easy bisection. >> > > I don't quite understand this. Yes building all commits is desirable, but > in the case of Phabricator each revision is going to be a single commit, > no? So why is this an issue? Or is it an issue only for github PRs? The problem is that many contributors, including Simon PJ, Richard, and me, tend to push batches of work. For instance, when I land contributors' differentials I first apply a batch, then validate locally, and then push as a chunk. We can change this if necessary, but it will need to be via social convention which hasn't worked very well historically. > I thought we had decided (somewhere, I forget where) that we were going to > require PRs to be squashed before pushing? > Please correct me if I'm forgetting something but I do not believe we have even decided that we would start accepting PRs for larger patches. I had said during ICFP that I was open to the idea, but experience with GitHub's reviewing tools has since led me to view the proposal with a bit more skepticism. Since this was prior to the creation of this mailing list I summarised these concerns in a private email to Manuel; I will forward this message to the list in a moment. Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From ben at well-typed.com Wed Nov 22 19:55:10 2017 From: ben at well-typed.com (Ben Gamari) Date: Wed, 22 Nov 2017 14:55:10 -0500 Subject: [GHC DevOps Group] [fwd] GitHub code review infrastructure Message-ID: <87vai26qch.fsf@ben-laptop.smart-cactus.org> An embedded and charset-unspecified text was scrubbed... Name: not available URL: -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: not available URL: -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From marlowsd at gmail.com Thu Nov 23 09:33:41 2017 From: marlowsd at gmail.com (Simon Marlow) Date: Thu, 23 Nov 2017 09:33:41 +0000 Subject: [GHC DevOps Group] CircleCI job accounting question In-Reply-To: <874lpm86ra.fsf@ben-laptop.smart-cactus.org> References: <8760agnulr.fsf@ben-laptop.smart-cactus.org> <0B8947A7-CF4E-4803-828E-2DC1FBC6F4F0@tweag.io> <8760aen9ov.fsf@ben-laptop.smart-cactus.org> <54496A02-05EE-43EF-B717-612829628D90@tweag.io> <87zi7njmth.fsf@ben-laptop.smart-cactus.org> <87mv3f80xv.fsf@ben-laptop.smart-cactus.org> <87d14a8h4s.fsf@ben-laptop.smart-cactus.org> <874lpm86ra.fsf@ben-laptop.smart-cactus.org> Message-ID: On 22 November 2017 at 19:15, Ben Gamari wrote: > Simon Marlow writes: > > > On 22 November 2017 at 15:31, Ben Gamari wrote: > > > >> Manuel M T Chakravarty writes: > >> > Why do we need the intermediate builds exactly? Wouldn’t they usually > >> > fail? (When I do PRs with multiple commits, the state of the tree > >> > between this commits will usually not be well-defined.) > >> > >> No, every commit should build. This is in part a difference between > >> Phabricator's patch-based model and GitHub's feature branch model. > >> However, many projects using the latter also demand that all > >> intermediate commits must be atomic, buildable changes. Sacrificing this > >> property greatly complicates bisection. > >> > >> Building all intermediates is desireable as ultimately we would like to > >> preserve per-commit build artifacts for the last few months of commits > >> to enable easy bisection. > >> > > > > I don't quite understand this. Yes building all commits is desirable, but > > in the case of Phabricator each revision is going to be a single commit, > > no? So why is this an issue? Or is it an issue only for github PRs? > > The problem is that many contributors, including Simon PJ, Richard, and > me, tend to push batches of work. For instance, when I land > contributors' differentials I first apply a batch, then validate > locally, and then push as a chunk. We can change this if necessary, but > it will need to be via social convention which hasn't worked very well > historically. > Ah, I see, I thought the problem was just to do with branches. Thanks for the clarification. > I thought we had decided (somewhere, I forget where) that we were going to > > require PRs to be squashed before pushing? > > > Please correct me if I'm forgetting something but I do not believe we > have even decided that we would start accepting PRs for larger patches. > I had said during ICFP that I was open to the idea, but experience > with GitHub's reviewing tools has since led me to view the proposal with > a bit more skepticism. Since this was prior to the creation of this > mailing list I summarised these concerns in a private email to Manuel; I > will forward this message to the list in a moment. > Right, we certainly haven't decided to move code reviewing to GitHub (and I also have deep concerns about that). I was thinking about the case where someone submits a PR via GitHub and we convert it into a Phabricator diff, squashing in the process, or if the PR doesn't require reviewing then we squash before pushing. Cheers Simon -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben at well-typed.com Thu Nov 23 15:32:27 2017 From: ben at well-typed.com (Ben Gamari) Date: Thu, 23 Nov 2017 10:32:27 -0500 Subject: [GHC DevOps Group] CircleCI job accounting question In-Reply-To: References: <8760agnulr.fsf@ben-laptop.smart-cactus.org> <0B8947A7-CF4E-4803-828E-2DC1FBC6F4F0@tweag.io> <8760aen9ov.fsf@ben-laptop.smart-cactus.org> <54496A02-05EE-43EF-B717-612829628D90@tweag.io> <87zi7njmth.fsf@ben-laptop.smart-cactus.org> <87mv3f80xv.fsf@ben-laptop.smart-cactus.org> <87d14a8h4s.fsf@ben-laptop.smart-cactus.org> <874lpm86ra.fsf@ben-laptop.smart-cactus.org> Message-ID: <87o9nt6mf3.fsf@ben-laptop.smart-cactus.org> Simon Marlow writes: > On 22 November 2017 at 19:15, Ben Gamari wrote: > >> Please correct me if I'm forgetting something but I do not believe we >> have even decided that we would start accepting PRs for larger patches. >> I had said during ICFP that I was open to the idea, but experience >> with GitHub's reviewing tools has since led me to view the proposal with >> a bit more skepticism. Since this was prior to the creation of this >> mailing list I summarised these concerns in a private email to Manuel; I >> will forward this message to the list in a moment. >> > > Right, we certainly haven't decided to move code reviewing to GitHub (and I > also have deep concerns about that). I was thinking about the case where > someone submits a PR via GitHub and we convert it into a Phabricator diff, > squashing in the process, or if the PR doesn't require reviewing then we > squash before pushing. > Right, I usually squash such PRs when I merge anyways. That being said, I also generally merge them along with other work. Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From ben at well-typed.com Thu Nov 23 18:45:50 2017 From: ben at well-typed.com (Ben Gamari) Date: Thu, 23 Nov 2017 13:45:50 -0500 Subject: [GHC DevOps Group] CircleCI question References: <8760agnulr.fsf@ben-laptop.smart-cactus.org> Message-ID: <87ine07s14.fsf@ben-laptop.smart-cactus.org> Hi Jonas, I'm hoping that you can perhaps shed light on the status of this build [1] as the interface is a bit perplexing. All at once the interface suggests that: * The build has "failed" and finished 23 minutes ago * "There was an issue while running this container and it was rerun." Note the use of past tense, perhaps suggesting that the build was rerun and finished? The second sentence is quite mysterious; is the "most recent run" the failed run or the * "The most recent run is shown." Presumably this is the re-run. * The build is still running. I honestly am not sure what I am supposed to do with information. It sounds as though there was an infrastructure issue on their end, causing the last attempted build to fail and resulting in a rerun, which is still ongoing. However, the email notification they sent claimed that there was an unexpected exit code (seemingly during the test step): ------ Failed ------ Uh-oh, this build did not succeed! ... ---------------- Failing command: ---------------- Command: Error executing build steps Exit code: n/a ------ Output ------ Unexpected error running build task: exit status 255 ----------------------------------------- The rest of your commands were successful ----------------------------------------- Spin up Environment Checkout code prepare-system submodules Boot Configure Build In general I'm a bit confused as to why they are notifying us of spurious build failures. Have you seen this sort of behavior before? Does the above interpretation sound correct? Is there any way to reduce the volume of notifications to *only* actual failed builds? Cheers, - Ben [1] https://circleci.com/gh/ghc/ghc/389 Screenshot here: http://home.smart-cactus.org/~ben/circleci.png -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From fuuzetsu at fuuzetsu.co.uk Thu Nov 23 19:27:09 2017 From: fuuzetsu at fuuzetsu.co.uk (Mateusz Kowalczyk) Date: Thu, 23 Nov 2017 19:27:09 +0000 Subject: [GHC DevOps Group] CircleCI question In-Reply-To: <87ine07s14.fsf@ben-laptop.smart-cactus.org> References: <8760agnulr.fsf@ben-laptop.smart-cactus.org> <87ine07s14.fsf@ben-laptop.smart-cactus.org> Message-ID: <04fd1c29-7eb5-74ea-4be4-c3821c9cfce9@fuuzetsu.co.uk> Hi Ben, Jonas is on holiday until Monday so his reachability is limited. Having said that… On 11/23/2017 06:45 PM, Ben Gamari wrote: > > Hi Jonas, > > I'm hoping that you can perhaps shed light on the status of this build > [1] as the interface is a bit perplexing. All at once the interface > suggests that: > > * The build has "failed" and finished 23 minutes ago > > * "There was an issue while running this container and it was rerun." > Note the use of past tense, perhaps > suggesting that the build was rerun and finished? The second sentence > is quite mysterious; is the "most recent run" the failed run or the > > * "The most recent run is shown." Presumably this is the re-run. > > * The build is still running. > > I honestly am not sure what I am supposed to do with information. > It sounds as though there was an infrastructure issue on their end, > causing the last attempted build to fail and resulting in a rerun, which > is still ongoing. However, the email notification they sent claimed that > there was an unexpected exit code (seemingly during the test step): I had a ticket about this with CircleCI. They have told me after a week-ish that it was fixed on their end and I did not see any new builds with this problem. Perhaps it's a regression. The ticket number in their system is 26023. Should I raise a new one or you want to? > ------ > Failed > ------ > > Uh-oh, this build did not succeed! > > ... > > ---------------- > Failing command: > ---------------- > Command: Error executing build steps > Exit code: n/a > > ------ > Output > ------ > Unexpected error running build task: exit status 255 > > > ----------------------------------------- > The rest of your commands were successful > ----------------------------------------- > > Spin up Environment > Checkout code > prepare-system > submodules > Boot > Configure > Build > > In general I'm a bit confused as to why they are notifying us of > spurious build failures. > > Have you seen this sort of behavior before? Does the above > interpretation sound correct? Is there any way to reduce the volume of > notifications to *only* actual failed builds? I'm not sure about notifications. > Cheers, > > - Ben > > > [1] https://circleci.com/gh/ghc/ghc/389 > Screenshot here: http://home.smart-cactus.org/~ben/circleci.png > -- Mateusz K. From jonas.chevalier at tweag.io Sun Nov 26 13:34:37 2017 From: jonas.chevalier at tweag.io (Jonas Chevalier) Date: Sun, 26 Nov 2017 13:34:37 +0000 Subject: [GHC DevOps Group] CircleCI question In-Reply-To: <04fd1c29-7eb5-74ea-4be4-c3821c9cfce9@fuuzetsu.co.uk> References: <8760agnulr.fsf@ben-laptop.smart-cactus.org> <87ine07s14.fsf@ben-laptop.smart-cactus.org> <04fd1c29-7eb5-74ea-4be4-c3821c9cfce9@fuuzetsu.co.uk> Message-ID: I can only confirm that I saw the same confusing behaviour: the build is marked as failed but is still running. In all of the cases the failure was infrastructure related and wad retried automatically. Once the second run finished, the build status gets overwritten. It looks like a design issue, it's not fatal but is definitely annoying. On Thu, 23 Nov 2017, 19:27 Mateusz Kowalczyk wrote: > Hi Ben, > > Jonas is on holiday until Monday so his reachability is limited. Having > said that… > > On 11/23/2017 06:45 PM, Ben Gamari wrote: > > > > Hi Jonas, > > > > I'm hoping that you can perhaps shed light on the status of this build > > [1] as the interface is a bit perplexing. All at once the interface > > suggests that: > > > > * The build has "failed" and finished 23 minutes ago > > > > * "There was an issue while running this container and it was rerun." > > Note the use of past tense, perhaps > > suggesting that the build was rerun and finished? The second sentence > > is quite mysterious; is the "most recent run" the failed run or the > > > > * "The most recent run is shown." Presumably this is the re-run. > > > > * The build is still running. > > > > I honestly am not sure what I am supposed to do with information. > > It sounds as though there was an infrastructure issue on their end, > > causing the last attempted build to fail and resulting in a rerun, which > > is still ongoing. However, the email notification they sent claimed that > > there was an unexpected exit code (seemingly during the test step): > > I had a ticket about this with CircleCI. They have told me after a > week-ish that it was fixed on their end and I did not see any new builds > with this problem. Perhaps it's a regression. The ticket number in their > system is 26023. Should I raise a new one or you want to? > > > ------ > > Failed > > ------ > > > > Uh-oh, this build did not succeed! > > > > ... > > > > ---------------- > > Failing command: > > ---------------- > > Command: Error executing build steps > > Exit code: n/a > > > > ------ > > Output > > ------ > > Unexpected error running build task: exit status 255 > > > > > > ----------------------------------------- > > The rest of your commands were successful > > ----------------------------------------- > > > > Spin up Environment > > Checkout code > > prepare-system > > submodules > > Boot > > Configure > > Build > > > > In general I'm a bit confused as to why they are notifying us of > > spurious build failures. > > > > Have you seen this sort of behavior before? Does the above > > interpretation sound correct? Is there any way to reduce the volume of > > notifications to *only* actual failed builds? > > I'm not sure about notifications. > > > Cheers, > > > > - Ben > > > > > > [1] https://circleci.com/gh/ghc/ghc/389 > > Screenshot here: http://home.smart-cactus.org/~ben/circleci.png > > > > > -- > Mateusz K. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jonas.chevalier at tweag.io Sun Nov 26 13:42:04 2017 From: jonas.chevalier at tweag.io (Jonas Chevalier) Date: Sun, 26 Nov 2017 13:42:04 +0000 Subject: [GHC DevOps Group] [fwd] GitHub code review infrastructure In-Reply-To: <87vai26qch.fsf@ben-laptop.smart-cactus.org> References: <87vai26qch.fsf@ben-laptop.smart-cactus.org> Message-ID: Hi Ben, Just some quick workarounds: It's possible to click on a line of code and paste it into a comment, that could act as a replacement for your pointers? In regards to diff changes, try adding ?ws=1 to the URL to get a diff without the whitespace, in some cases it might help. Both are not ideal solutions but might alleviate those issues. Best, Jonas On Wed, 22 Nov 2017, 19:56 Ben Gamari wrote: > > Attached is a message which I wrote to Manuel shortly after ICFP > relating some recent experiences with GitHub's code review > functionality. > > Since I wrote this there are a few additional issues that I stumbled > upon write reviewing other patches which I would like to mention, > > * It is not possible to comment on lines that are not in the vicinity of > a change. I often find myself doing this to point out parts of the > code that a contributor may have overlooked. > > * I would reiterate the difficulty of keeping ones bearings in a large > patch. Recently I reviewed Moritz's Relocatable builds patch [1] > and found myself frequently wondering what file I was looking at. > While in Differential I can simply cast a glance to the file tree, in > GitHub I need to scroll up until I spot the beginning of the file, > and then find where I was previously and resume. > > * I have found that GitHub is quite conservative in pointing out > similarities between old and new code which tends to obscure what > should be obvious preserved structure. To take a recent example: see > GitHub's rendering [2] of a piece of Alan's recent TTG work and try > to work out what that hunk is doing. Now look at the same hunk on > Phabricator [3]. The difference is in my opinion far more obvious. > > All of these papercuts add up and have a real cost. If the consensus > really is that we want to move review to GitHub then I will certainly go > along, but I do feel that it would be a significant regression. > > I have still not yet tried using Reviewable.io, so I can't say with > certainty whether it would address these concerns. > > Cheers, > > - Ben > > > [1] https://github.com/snowleopard/hadrian/pull/445/files > [2] > https://github.com/ghc/ghc/commit/47ad6578ea460999b53eb4293c3a3b3017a56d65#diff-df7abd11782680fc55c1d1b04747dba9R1103 > [3] > https://phabricator.haskell.org/rGHC47ad6578ea460999b53eb4293c3a3b3017a56d65#C8856OL84 > > > -------------------- Start of forwarded message -------------------- > From: Ben Gamari > To: Manuel Chakravarty > Subject: GitHub code review infrastructure > BCC: ben at smart-cactus.org > Date: Thu, 28 Sep 2017 09:20:20 -0400 > Message-ID: <87y3ozrmcr.fsf at ben-laptop.smart-cactus.org> > > Hi Manuel, > > Since we spoke I've been working with a contributor who submitted a > somewhat non-trivial change [1] via pull request. This was my first > non-trivial code review with on GitHub with myself in the role of > reviewer. By GHC standards the patch is neither large nor subtle; it's > largely just me guiding a new contributor through the codebase. > > I'll admit that after this experience I'm a bit less rosy on GitHub's > code review functionality than I was on Monday. I thought it might be > helpful (for me, if not also for you) to summarize my impressions: > > * Commenting works well although quoting bits of comments is a bit > inconvenient. > > * The lack of an overview of the diff (like Differential's "file tree" > pane) makes reviewing patches of moderate-to-large size rather tricky > > * The treatment of rebased branches is truly awful. The previous > commits appear to be totally gone and cannot be viewed. This is > in contrast to Phabricator, where you can view each revision of > a differential, which in the past has been very helpful when we > are trying to reconstruct the reasoning behind a change. > > * While the discussion on pre-rebase commits is preserved, it is > impossible to view it in the context in which it was made since the > patches themselves are gone. > > * The "Conversation" interface gets rather hard to follow after a short > while since there are free-standing comments, new commit > notifications, and inline code comments in a low-contrast, > low-whitespace wall of boxes. > > Some of these issues above may be just a function of my unfamiliarity > with the interface. Many of the other issues could no doubt be worked > around with a combination of Greasemonkey and custom CSS. That being > said, the fact that this may be necessary is a bit unfortunate and the > more fundamental issues (inability to view previously revisions, > treatment of rebases) can't be worked around. > > This is a shame since, as I related on Monday, I think the arcanist > workflow introduces a lot of friction. Relying on Git, a tool nearly all > our users are familiar with, for patch management makes much more sense > IMHO. However, I'll admit I was hoping that GitHub's code review > experience was better than it is. > > Perhaps reviewable.io or some alternative would be better? > > Anyways, these are just my two-pence. I'm still happy to consider moving > and am not ruling out that on the whole it still may be the right thing > to do; however, I was just a bit surprised by some of the issues that I > encountered and thought it would only be fair to let you know. > > Cheers, > > - Ben > > > [1] https://github.com/ghc/ghc/pull/76/files > -------------------- End of forwarded message -------------------- > _______________________________________________ > Ghc-devops-group mailing list > Ghc-devops-group at haskell.org > https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devops-group > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben at well-typed.com Wed Nov 29 20:54:09 2017 From: ben at well-typed.com (Ben Gamari) Date: Wed, 29 Nov 2017 15:54:09 -0500 Subject: [GHC DevOps Group] CircleCI job accounting question In-Reply-To: <8760agnulr.fsf@ben-laptop.smart-cactus.org> References: <8760agnulr.fsf@ben-laptop.smart-cactus.org> Message-ID: <87inds9578.fsf@ben-laptop.smart-cactus.org> Ben Gamari writes: > Hi Manuel, > > I seem to recall you earlier mentioning that CircleCI jobs associated > with forks of a repository do not count towards the job limit of the > forked repository. How certain are you that this is the case? > It turns out that what I suspected is indeed the case: my builds are indeed running against the ghc organization's container allowance. This can be seen by clicking on the "Show Queued Builds" button on a queued build. The message above the queue list says, This build has been queued behind the following builds for 05:19 This build is running under ghc's plan which provides 1 containers, plus 3 additional containers available for free and open source projects. Add Containers to run more builds concurrently. Unfortunately I haven't yet been able to determine /why/ this is so. I suspect it is because I am a member of the `ghc` organization, but I haven't been able to find any clear language in the documentation confirming that this is so. Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: