From manuel.chakravarty at tweag.io Tue Jun 5 13:32:33 2018 From: manuel.chakravarty at tweag.io (Manuel M T Chakravarty) Date: Tue, 5 Jun 2018 23:32:33 +1000 Subject: [GHC DevOps Group] State of CI Message-ID: Hi Ben, I just wanted to touch base regarding the state of the GHC CI effort. As far as I am aware, we have CI running on both CircleCI and Appveyor (with Google generously donating the build machines). Is that right? Do these builds also generate complete build artefacts by now? (We wanted to eventually generate everything including documentation automatically.) If I am not mistaken, we still can’t run CircleCI on Phab Diffs. Moreover, there was some noise that Phabricator might be changing their business model, which might make it less attractive for GHC (but I am not sure about the details). Is that correct? I will be giving a talk at ZuriHac and would like to know the open problems, so I can try to motivate attendants to help us and contribute. Cheers, Manuel From ben at well-typed.com Tue Jun 5 16:30:26 2018 From: ben at well-typed.com (Ben Gamari) Date: Tue, 05 Jun 2018 12:30:26 -0400 Subject: [GHC DevOps Group] State of CI In-Reply-To: References: Message-ID: <87zi09gpr6.fsf@smart-cactus.org> Manuel M T Chakravarty writes: > Hi Ben, > Hi Manuel, > I just wanted to touch base regarding the state of the GHC CI effort. > > As far as I am aware, we have CI running on both CircleCI and Appveyor > (with Google generously donating the build machines). Is that right? > That is right. Alp and I have been steadily chipping away at the remaining build issues but otherwise things seem to be working well. > Do these builds also generate complete build artefacts by now? (We > wanted to eventually generate everything including documentation > automatically.) > > If I am not mistaken, we still can’t run CircleCI on Phab Diffs. > Moreover, there was some noise that Phabricator might be changing > their business model, which might make it less attractive for GHC (but > I am not sure about the details). Is that correct? > That is correct. Phacility is moving to an explicitly pay-to-play model; the source is available, but they aren't accepting patches and opening tickets requires a support contract. This isn't the end of the world for us, but it certainly makes Phabricator less attractive in the long-run. However, given the recent GitHub news, I'm not sure this is a terribly attractive option either. All of this certainly complicates the CI story. On one hand, I've been a tad reluctant to spend too much time hacking Phabricator/CircleCI integration together given the Phabricator situation. On the other hand, CircleCI and Appveyor both only support GitHub, so a move to, for instance, GitLab doesn't really unblock us. For the time being I would say we should probably continue pushing ahead with Phabricator. It likely won't be too hard to get something working and it will finally allow us to begin moving away from Harbormaster. Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From simonpj at microsoft.com Tue Jun 5 20:51:51 2018 From: simonpj at microsoft.com (Simon Peyton Jones) Date: Tue, 5 Jun 2018 20:51:51 +0000 Subject: [GHC DevOps Group] State of CI In-Reply-To: <87zi09gpr6.fsf@smart-cactus.org> References: <87zi09gpr6.fsf@smart-cactus.org> Message-ID: I have no inside knowledge, but I suspect that Microsoft's acquisition of Github means - that GitHub will be largely undisturbed culturally - that GitHub will have more oomph behind it, so it'll become yet more the de-facto choice than it already is | That is correct. Phacility is moving to an explicitly pay-to-play model; | CircleCI and Appveyor both only support GitHub, I know nothing of the nitty-gritty reality, but from what you say about Phab, it sounds to me as if the wind is blowing us toward GitHub even if it doesn’t do everything we might want. Simon | -----Original Message----- | From: Ghc-devops-group On Behalf | Of Ben Gamari | Sent: 05 June 2018 17:30 | To: Manuel M T Chakravarty | Cc: ghc-devops-group at haskell.org | Subject: Re: [GHC DevOps Group] State of CI | | Manuel M T Chakravarty writes: | | > Hi Ben, | > | Hi Manuel, | | > I just wanted to touch base regarding the state of the GHC CI effort. | > | > As far as I am aware, we have CI running on both CircleCI and Appveyor | > (with Google generously donating the build machines). Is that right? | > | That is right. Alp and I have been steadily chipping away at the | remaining build issues but otherwise things seem to be working well. | | > Do these builds also generate complete build artefacts by now? (We | > wanted to eventually generate everything including documentation | > automatically.) | > | | | > If I am not mistaken, we still can’t run CircleCI on Phab Diffs. | > Moreover, there was some noise that Phabricator might be changing | > their business model, which might make it less attractive for GHC (but | > I am not sure about the details). Is that correct? | > | That is correct. Phacility is moving to an explicitly pay-to-play model; | the source is available, but they aren't accepting patches and opening | tickets requires a support contract. This isn't the end of the world for | us, but it certainly makes Phabricator less attractive in the long-run. | However, given the recent GitHub news, I'm not sure this is a terribly | attractive option either. | | All of this certainly complicates the CI story. On one hand, I've been | a tad reluctant to spend too much time hacking Phabricator/CircleCI | integration together given the Phabricator situation. On the other hand, | CircleCI and Appveyor both only support GitHub, so a move to, for | instance, GitLab doesn't really unblock us. | | For the time being I would say we should probably continue pushing ahead | with Phabricator. It likely won't be too hard to get something working | and it will finally allow us to begin moving away from Harbormaster. | | Cheers, | | - Ben From gershomb at gmail.com Wed Jun 6 01:23:59 2018 From: gershomb at gmail.com (Gershom B) Date: Tue, 5 Jun 2018 21:23:59 -0400 Subject: [GHC DevOps Group] State of CI In-Reply-To: References: <87zi09gpr6.fsf@smart-cactus.org> Message-ID: On Tue, Jun 5, 2018 at 4:51 PM, Simon Peyton Jones wrote: > I have no inside knowledge, but I suspect that Microsoft's acquisition of Github means > - that GitHub will be largely undisturbed culturally > - that GitHub will have more oomph behind it, so it'll become yet > more the de-facto choice than it already is For what it's worth, the latter is definitely not happening. People are discussing migrating away from it out of concern with regards to the direction microsoft may evolve it, and in recognition that it is better to avoid lock-in with regards to proprietary systems (in terms of trackers, etc) now rather than later, and while export is available, as there are concerns that it might not be forever (not that this is necessarily a valid concern or not -- but it is certainly driving migration). Some metrics have shown that there has been a marked increase in imports of github accounts to gitlab in the past day or so. Certainly the expectation is that microsoft will evolve github in _some_ direction under its management, and this uncertainty enough is sufficient to drive some migration -- and should give pause to GHC plans as well -- it doesn't seem like a savvy move to cut over to something that could well be a moving target until some dust settles. Cheers, Gershom From manuel.chakravarty at tweag.io Wed Jun 6 05:23:48 2018 From: manuel.chakravarty at tweag.io (Manuel M T Chakravarty) Date: Wed, 6 Jun 2018 15:23:48 +1000 Subject: [GHC DevOps Group] State of CI In-Reply-To: <87zi09gpr6.fsf@smart-cactus.org> References: <87zi09gpr6.fsf@smart-cactus.org> Message-ID: <8D5F4D4B-6D78-4AB3-9B1F-D077D971040D@tweag.io> Thanks for the update! Manuel > Am 06.06.2018 um 02:30 schrieb Ben Gamari : > > Manuel M T Chakravarty writes: > >> Hi Ben, >> > Hi Manuel, > >> I just wanted to touch base regarding the state of the GHC CI effort. >> >> As far as I am aware, we have CI running on both CircleCI and Appveyor >> (with Google generously donating the build machines). Is that right? >> > That is right. Alp and I have been steadily chipping away at the > remaining build issues but otherwise things seem to be working well. > >> Do these builds also generate complete build artefacts by now? (We >> wanted to eventually generate everything including documentation >> automatically.) >> > > >> If I am not mistaken, we still can’t run CircleCI on Phab Diffs. >> Moreover, there was some noise that Phabricator might be changing >> their business model, which might make it less attractive for GHC (but >> I am not sure about the details). Is that correct? >> > That is correct. Phacility is moving to an explicitly pay-to-play model; > the source is available, but they aren't accepting patches and opening > tickets requires a support contract. This isn't the end of the world for > us, but it certainly makes Phabricator less attractive in the long-run. > However, given the recent GitHub news, I'm not sure this is a terribly > attractive option either. > > All of this certainly complicates the CI story. On one hand, I've been > a tad reluctant to spend too much time hacking Phabricator/CircleCI > integration together given the Phabricator situation. On the other hand, > CircleCI and Appveyor both only support GitHub, so a move to, for > instance, GitLab doesn't really unblock us. > > For the time being I would say we should probably continue pushing ahead > with Phabricator. It likely won't be too hard to get something working > and it will finally allow us to begin moving away from Harbormaster. > > Cheers, > > - Ben From manuel.chakravarty at tweag.io Wed Jun 6 06:12:05 2018 From: manuel.chakravarty at tweag.io (Manuel M T Chakravarty) Date: Wed, 6 Jun 2018 16:12:05 +1000 Subject: [GHC DevOps Group] Phacility -> Reviewable [was: Re: State of CI] In-Reply-To: References: <87zi09gpr6.fsf@smart-cactus.org> Message-ID: > Am 06.06.2018 um 06:51 schrieb Simon Peyton Jones : > > I have no inside knowledge, but I suspect that Microsoft's acquisition of Github means > - that GitHub will be largely undisturbed culturally > - that GitHub will have more oomph behind it, so it'll become yet > more the de-facto choice than it already is I agree, Simon. Microsoft 2018 seems to have different priorities from what it used to have — that is clear just from looking at the sheer scale of open-source contributions. I think, the main obstacle here is perception. Especially among the Linux community, there are many people who historically distrust Microsoft. We have seen this discussion on the GHC list before, where some individuals refuse to even consider GitHub just because not every last bit of their software is open source. Alas, I predict that these people will feel validate by the acquisition. > | That is correct. Phacility is moving to an explicitly pay-to-play model; > | CircleCI and Appveyor both only support GitHub, > > I know nothing of the nitty-gritty reality, but from what you say about Phab, it sounds to me as if the wind is blowing us toward GitHub even if it doesn’t do everything we might want. This is my impression, too. Now, I see no problem in paying for a service when it is useful — as in we get better support for code reviews than what GitHub provides for free. However, if we are going to pay-to-play anyway, then I would strongly suggest to consider services like https://reviewable.io , which integrate well with GitHub, and hence with CircleCI and Appveyor, with no extra work on our side. In fact, I just looked at >’s pricing and for open source projects it seems to be free. Ben & SimonM, could you please have a closer look at whether https://reviewable.io addresses the issues that you have got with GitHub’s code reviews? (I notice that one point Ben has made, namely how comments at specific lines are handled by GitHub, is something that https://reviewable.io explicitly claims they do much better.) Signing up for the free option is trivial with your GitHub account and you can link the GHC org from there. We can try https://reviewable.io on individual PRs without enabling it for the entire repo to get a feel for it. I’d like to know if there is any strong reason for not replacing Phacility by Reviewable. Thanks, Manuel > Simon > > | -----Original Message----- > | From: Ghc-devops-group On Behalf > | Of Ben Gamari > | Sent: 05 June 2018 17:30 > | To: Manuel M T Chakravarty > | Cc: ghc-devops-group at haskell.org > | Subject: Re: [GHC DevOps Group] State of CI > | > | Manuel M T Chakravarty writes: > | > | > Hi Ben, > | > > | Hi Manuel, > | > | > I just wanted to touch base regarding the state of the GHC CI effort. > | > > | > As far as I am aware, we have CI running on both CircleCI and Appveyor > | > (with Google generously donating the build machines). Is that right? > | > > | That is right. Alp and I have been steadily chipping away at the > | remaining build issues but otherwise things seem to be working well. > | > | > Do these builds also generate complete build artefacts by now? (We > | > wanted to eventually generate everything including documentation > | > automatically.) > | > > | > | > | > If I am not mistaken, we still can’t run CircleCI on Phab Diffs. > | > Moreover, there was some noise that Phabricator might be changing > | > their business model, which might make it less attractive for GHC (but > | > I am not sure about the details). Is that correct? > | > > | That is correct. Phacility is moving to an explicitly pay-to-play model; > | the source is available, but they aren't accepting patches and opening > | tickets requires a support contract. This isn't the end of the world for > | us, but it certainly makes Phabricator less attractive in the long-run. > | However, given the recent GitHub news, I'm not sure this is a terribly > | attractive option either. > | > | All of this certainly complicates the CI story. On one hand, I've been > | a tad reluctant to spend too much time hacking Phabricator/CircleCI > | integration together given the Phabricator situation. On the other hand, > | CircleCI and Appveyor both only support GitHub, so a move to, for > | instance, GitLab doesn't really unblock us. > | > | For the time being I would say we should probably continue pushing ahead > | with Phabricator. It likely won't be too hard to get something working > | and it will finally allow us to begin moving away from Harbormaster. > | > | Cheers, > | > | - Ben -------------- next part -------------- An HTML attachment was scrubbed... URL: From manuel.chakravarty at tweag.io Wed Jun 6 06:22:17 2018 From: manuel.chakravarty at tweag.io (Manuel M T Chakravarty) Date: Wed, 6 Jun 2018 16:22:17 +1000 Subject: [GHC DevOps Group] State of CI In-Reply-To: References: <87zi09gpr6.fsf@smart-cactus.org> Message-ID: <0E76E105-11E2-406F-81B2-C8D1ACB11117@tweag.io> IMHO, this is the usual huffing and puffing of the Internet. In a few weeks everybody will have forgotten about it and move on to the next ”threat”. Cheers, Manuel > Am 06.06.2018 um 11:23 schrieb Gershom B : > > On Tue, Jun 5, 2018 at 4:51 PM, Simon Peyton Jones > wrote: >> I have no inside knowledge, but I suspect that Microsoft's acquisition of Github means >> - that GitHub will be largely undisturbed culturally >> - that GitHub will have more oomph behind it, so it'll become yet >> more the de-facto choice than it already is > > For what it's worth, the latter is definitely not happening. People > are discussing migrating away from it out of concern with regards to > the direction microsoft may evolve it, and in recognition that it is > better to avoid lock-in with regards to proprietary systems (in terms > of trackers, etc) now rather than later, and while export is > available, as there are concerns that it might not be forever (not > that this is necessarily a valid concern or not -- but it is certainly > driving migration). Some metrics have shown that there has been a > marked increase in imports of github accounts to gitlab in the past > day or so. > > Certainly the expectation is that microsoft will evolve github in > _some_ direction under its management, and this uncertainty enough is > sufficient to drive some migration -- and should give pause to GHC > plans as well -- it doesn't seem like a savvy move to cut over to > something that could well be a moving target until some dust settles. > > Cheers, > Gershom > _______________________________________________ > Ghc-devops-group mailing list > Ghc-devops-group at haskell.org > https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devops-group From gershomb at gmail.com Wed Jun 6 07:14:16 2018 From: gershomb at gmail.com (Gershom B) Date: Wed, 6 Jun 2018 00:14:16 -0700 Subject: [GHC DevOps Group] Phacility -> Reviewable [was: Re: State of CI] In-Reply-To: References: <87zi09gpr6.fsf@smart-cactus.org> Message-ID: I really don’t want to act like a tech-pundit here, but I think there’s an important point worth considering which is perhaps too business-y to be noticed in the noise around the acquisition. And this is not about microsoft having better or worse priorities, etc. We can recognize the good work MS has produced and funded in the past years, and their changed relationship to free software licenses, etc. and also recognize that there are going to be business decisions at work that are almost independent of who acquired github. Github, like most tech startups, was operating at a loss for years. Honestly, despite their paid services, the business model was to give a bunch of stuff away for free or below cost and make it really nice, to generate a huge user-base. Having established a dominant share of users, you then exit by selling off the company, and the users, who are now somewhat locked-into the ecosystem. That’s what happened. The issue is not about values or anything related. It is simply in the nature of the business model, and in fact is the whole reason github managed to be so well capitalized despite operating at a loss to begin with. (In fact, the paid services were not actually about making money — they were proofs of concept to illustrate how one could leverage the position of github to make money. The model wasn’t to make money — it was to show how one could make money after acquiring github.) However, now that github is acquired — and it doesn’t matter particularly that it was microsoft rather than ibm or oracle or SAP — there’s no incentive to give stuff away below-cost just to foster growth in users. And there’s no incentive to cater to projects that aren’t of the sort that will pay significant sums for premium services. That’s not about corporate values or priorities, per se. That’s just business. As is the fate of all such acquisitions, github will have to be turned around from being in the red, but with many users, to being in the black. Part of that may be direct, and part may be in terms of focusing on integration with other microsoft products so as to steer github users into the broader ecosystem of paid microsoft tooling. (E.g., I expect that a great deal of development effort will be devoted to increased integration with visualstudio team services). That means changes will _have_ to occur. I don’t know at what pace, and I don’t know how drastic, but that’s just the financial realities. Maybe, at the end of the day, github will end up being more suitable, not less, for ghc and related dev, even if that means a certain degree of payment for premium services. But maybe not. What is certainly the case is that there will be some degree of growing pains, and if ghc is seeking something stable that reduces overhead, it does not seem to make sense to me to make a switch precisely at a point where things are most situated to have to suffer through these pains. Cheers, Gershom On June 6, 2018 at 2:12:29 AM, Manuel M T Chakravarty ( manuel.chakravarty at tweag.io) wrote: Am 06.06.2018 um 06:51 schrieb Simon Peyton Jones : I have no inside knowledge, but I suspect that Microsoft's acquisition of Github means - that GitHub will be largely undisturbed culturally - that GitHub will have more oomph behind it, so it'll become yet more the de-facto choice than it already is I agree, Simon. Microsoft 2018 seems to have different priorities from what it used to have — that is clear just from looking at the sheer scale of open-source contributions. I think, the main obstacle here is perception. Especially among the Linux community, there are many people who historically distrust Microsoft. We have seen this discussion on the GHC list before, where some individuals refuse to even consider GitHub just because not every last bit of their software is open source. Alas, I predict that these people will feel validate by the acquisition. | That is correct. Phacility is moving to an explicitly pay-to-play model; | CircleCI and Appveyor both only support GitHub, I know nothing of the nitty-gritty reality, but from what you say about Phab, it sounds to me as if the wind is blowing us toward GitHub even if it doesn’t do everything we might want. This is my impression, too. Now, I see no problem in paying for a service when it is useful — as in we get better support for code reviews than what GitHub provides for free. However, if we are going to pay-to-play anyway, then I would strongly suggest to consider services like https://reviewable.io, which integrate well with GitHub, and hence with CircleCI and Appveyor, with no extra work on our side. In fact, I just looked at ’s pricing and for open source projects it seems to be free. Ben & SimonM, could you please have a closer look at whether https://reviewable.io addresses the issues that you have got with GitHub’s code reviews? (I notice that one point Ben has made, namely how comments at specific lines are handled by GitHub, is something that https://reviewable.io explicitly claims they do much better.) Signing up for the free option is trivial with your GitHub account and you can link the GHC org from there. We can try https://reviewable.io on individual PRs without enabling it for the entire repo to get a feel for it. I’d like to know if there is any strong reason for not replacing Phacility by Reviewable. Thanks, Manuel Simon | -----Original Message----- | From: Ghc-devops-group On Behalf | Of Ben Gamari | Sent: 05 June 2018 17:30 | To: Manuel M T Chakravarty | Cc: ghc-devops-group at haskell.org | Subject: Re: [GHC DevOps Group] State of CI | | Manuel M T Chakravarty writes: | | > Hi Ben, | > | Hi Manuel, | | > I just wanted to touch base regarding the state of the GHC CI effort. | > | > As far as I am aware, we have CI running on both CircleCI and Appveyor | > (with Google generously donating the build machines). Is that right? | > | That is right. Alp and I have been steadily chipping away at the | remaining build issues but otherwise things seem to be working well. | | > Do these builds also generate complete build artefacts by now? (We | > wanted to eventually generate everything including documentation | > automatically.) | > | | | > If I am not mistaken, we still can’t run CircleCI on Phab Diffs. | > Moreover, there was some noise that Phabricator might be changing | > their business model, which might make it less attractive for GHC (but | > I am not sure about the details). Is that correct? | > | That is correct. Phacility is moving to an explicitly pay-to-play model; | the source is available, but they aren't accepting patches and opening | tickets requires a support contract. This isn't the end of the world for | us, but it certainly makes Phabricator less attractive in the long-run. | However, given the recent GitHub news, I'm not sure this is a terribly | attractive option either. | | All of this certainly complicates the CI story. On one hand, I've been | a tad reluctant to spend too much time hacking Phabricator/CircleCI | integration together given the Phabricator situation. On the other hand, | CircleCI and Appveyor both only support GitHub, so a move to, for | instance, GitLab doesn't really unblock us. | | For the time being I would say we should probably continue pushing ahead | with Phabricator. It likely won't be too hard to get something working | and it will finally allow us to begin moving away from Harbormaster. | | Cheers, | | - Ben _______________________________________________ Ghc-devops-group mailing list Ghc-devops-group at haskell.org https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devops-group -------------- next part -------------- An HTML attachment was scrubbed... URL: From manuel.chakravarty at tweag.io Wed Jun 6 08:18:24 2018 From: manuel.chakravarty at tweag.io (Manuel M T Chakravarty) Date: Wed, 6 Jun 2018 18:18:24 +1000 Subject: [GHC DevOps Group] Phacility -> Reviewable [was: Re: State of CI] In-Reply-To: References: <87zi09gpr6.fsf@smart-cactus.org> Message-ID: <34A477A7-428C-4B41-B6C9-7F9C47C44CDA@tweag.io> I appreciate your analysis of the VC funding model for building companies. However, rather than looking at generalities, I think, it makes more sense to look at specifics — for example that Microsoft has acquired Xamarin and HockeyApp a while ago and that both of these companies did offer a combination of free and paid services, just like GitHub. (I do use HockeyApp for Haskell for Mac btw.) With these companies, Microsoft has actually done exactly what they announced they will do with GitHub. Microsoft derives value from the integration of these services into their core products and from being able to sell services on top. There is no need for GitHub the service to pay the bills by pure GitHub subscriptions and pure GitHub enterprise deployments. (Nobody would expect, say, Apple’s Swift team to pay their own bills.) Overall, the future of Phacility seems much more unpredictable to me than the future of GitHub. After all, Phacility just literally demonstrated that they are happy to change the terms mid-flight. Hence, it appears prudent to move away from Phacility (which is all I am proposing). Cheers, Manuel > Am 06.06.2018 um 17:14 schrieb Gershom B : > > I really don’t want to act like a tech-pundit here, but I think there’s an important point worth considering which is perhaps too business-y to be noticed in the noise around the acquisition. And this is not about microsoft having better or worse priorities, etc. We can recognize the good work MS has produced and funded in the past years, and their changed relationship to free software licenses, etc. and also recognize that there are going to be business decisions at work that are almost independent of who acquired github. > > Github, like most tech startups, was operating at a loss for years. Honestly, despite their paid services, the business model was to give a bunch of stuff away for free or below cost and make it really nice, to generate a huge user-base. Having established a dominant share of users, you then exit by selling off the company, and the users, who are now somewhat locked-into the ecosystem. That’s what happened. The issue is not about values or anything related. It is simply in the nature of the business model, and in fact is the whole reason github managed to be so well capitalized despite operating at a loss to begin with. (In fact, the paid services were not actually about making money — they were proofs of concept to illustrate how one could leverage the position of github to make money. The model wasn’t to make money — it was to show how one could make money after acquiring github.) > > However, now that github is acquired — and it doesn’t matter particularly that it was microsoft rather than ibm or oracle or SAP — there’s no incentive to give stuff away below-cost just to foster growth in users. And there’s no incentive to cater to projects that aren’t of the sort that will pay significant sums for premium services. That’s not about corporate values or priorities, per se. That’s just business. As is the fate of all such acquisitions, github will have to be turned around from being in the red, but with many users, to being in the black. Part of that may be direct, and part may be in terms of focusing on integration with other microsoft products so as to steer github users into the broader ecosystem of paid microsoft tooling. (E.g., I expect that a great deal of development effort will be devoted to increased integration with visualstudio team services). That means changes will _have_ to occur. I don’t know at what pace, and I don’t know how drastic, but that’s just the financial realities. Maybe, at the end of the day, github will end up being more suitable, not less, for ghc and related dev, even if that means a certain degree of payment for premium services. But maybe not. What is certainly the case is that there will be some degree of growing pains, and if ghc is seeking something stable that reduces overhead, it does not seem to make sense to me to make a switch precisely at a point where things are most situated to have to suffer through these pains. > > Cheers, > Gershom > > > > > On June 6, 2018 at 2:12:29 AM, Manuel M T Chakravarty (manuel.chakravarty at tweag.io ) wrote: > >>> Am 06.06.2018 um 06:51 schrieb Simon Peyton Jones >: >>> >>> I have no inside knowledge, but I suspect that Microsoft's acquisition of Github means >>> - that GitHub will be largely undisturbed culturally >>> - that GitHub will have more oomph behind it, so it'll become yet >>> more the de-facto choice than it already is >> >> I agree, Simon. Microsoft 2018 seems to have different priorities from what it used to have — that is clear just from looking at the sheer scale of open-source contributions. >> >> I think, the main obstacle here is perception. Especially among the Linux community, there are many people who historically distrust Microsoft. We have seen this discussion on the GHC list before, where some individuals refuse to even consider GitHub just because not every last bit of their software is open source. Alas, I predict that these people will feel validate by the acquisition. >> >>> | That is correct. Phacility is moving to an explicitly pay-to-play model; >>> | CircleCI and Appveyor both only support GitHub, >>> >>> I know nothing of the nitty-gritty reality, but from what you say about Phab, it sounds to me as if the wind is blowing us toward GitHub even if it doesn’t do everything we might want. >> >> This is my impression, too. Now, I see no problem in paying for a service when it is useful — as in we get better support for code reviews than what GitHub provides for free. However, if we are going to pay-to-play anyway, then I would strongly suggest to consider services like https://reviewable.io , which integrate well with GitHub, and hence with CircleCI and Appveyor, with no extra work on our side. >> >> In fact, I just looked at >’s pricing and for open source projects it seems to be free. >> >> Ben & SimonM, could you please have a closer look at whether https://reviewable.io addresses the issues that you have got with GitHub’s code reviews? (I notice that one point Ben has made, namely how comments at specific lines are handled by GitHub, is something that https://reviewable.io explicitly claims they do much better.) >> >> Signing up for the free option is trivial with your GitHub account and you can link the GHC org from there. We can try https://reviewable.io on individual PRs without enabling it for the entire repo to get a feel for it. >> >> I’d like to know if there is any strong reason for not replacing Phacility by Reviewable. >> >> Thanks, >> Manuel >> >>> Simon >>> >>> | -----Original Message----- >>> | From: Ghc-devops-group > On Behalf >>> | Of Ben Gamari >>> | Sent: 05 June 2018 17:30 >>> | To: Manuel M T Chakravarty > >>> | Cc: ghc-devops-group at haskell.org >>> | Subject: Re: [GHC DevOps Group] State of CI >>> | >>> | Manuel M T Chakravarty > writes: >>> | >>> | > Hi Ben, >>> | > >>> | Hi Manuel, >>> | >>> | > I just wanted to touch base regarding the state of the GHC CI effort. >>> | > >>> | > As far as I am aware, we have CI running on both CircleCI and Appveyor >>> | > (with Google generously donating the build machines). Is that right? >>> | > >>> | That is right. Alp and I have been steadily chipping away at the >>> | remaining build issues but otherwise things seem to be working well. >>> | >>> | > Do these builds also generate complete build artefacts by now? (We >>> | > wanted to eventually generate everything including documentation >>> | > automatically.) >>> | > >>> | >>> | >>> | > If I am not mistaken, we still can’t run CircleCI on Phab Diffs. >>> | > Moreover, there was some noise that Phabricator might be changing >>> | > their business model, which might make it less attractive for GHC (but >>> | > I am not sure about the details). Is that correct? >>> | > >>> | That is correct. Phacility is moving to an explicitly pay-to-play model; >>> | the source is available, but they aren't accepting patches and opening >>> | tickets requires a support contract. This isn't the end of the world for >>> | us, but it certainly makes Phabricator less attractive in the long-run. >>> | However, given the recent GitHub news, I'm not sure this is a terribly >>> | attractive option either. >>> | >>> | All of this certainly complicates the CI story. On one hand, I've been >>> | a tad reluctant to spend too much time hacking Phabricator/CircleCI >>> | integration together given the Phabricator situation. On the other hand, >>> | CircleCI and Appveyor both only support GitHub, so a move to, for >>> | instance, GitLab doesn't really unblock us. >>> | >>> | For the time being I would say we should probably continue pushing ahead >>> | with Phabricator. It likely won't be too hard to get something working >>> | and it will finally allow us to begin moving away from Harbormaster. >>> | >>> | Cheers, >>> | >>> | - Ben >> >> _______________________________________________ >> Ghc-devops-group mailing list >> Ghc-devops-group at haskell.org >> https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devops-group -------------- next part -------------- An HTML attachment was scrubbed... URL: From m at tweag.io Wed Jun 6 08:21:34 2018 From: m at tweag.io (Boespflug, Mathieu) Date: Wed, 6 Jun 2018 10:21:34 +0200 Subject: [GHC DevOps Group] State of CI In-Reply-To: References: <87zi09gpr6.fsf@smart-cactus.org> Message-ID: Hi Gershom, On 6 June 2018 at 03:23, Gershom B wrote: > > On Tue, Jun 5, 2018 at 4:51 PM, Simon Peyton Jones > wrote: > > I have no inside knowledge, but I suspect that Microsoft's acquisition of Github means > > - that GitHub will be largely undisturbed culturally > > - that GitHub will have more oomph behind it, so it'll become yet > > more the de-facto choice than it already is > > For what it's worth, the latter is definitely not happening. [...] > Some metrics have shown that there has been a > marked increase in imports of github accounts to gitlab in the past > day or so. I assume you're referring to this: https://twitter.com/gitlab/status/1004143715844124673. I think it's premature to draw any inference from a single metric, let alone "definite" ones. Adoption is the result of the *net* inflow of users (new users minus lost users) integrated over time. With any substantial change, there are always going to be some disgruntled users, and others hedging their bets. That says little about the influx (or lack thereof) of new users and about the longer term trend. > Certainly the expectation is that microsoft will evolve github in > _some_ direction under its management, and this uncertainty enough is > sufficient to drive some migration -- and should give pause to GHC > plans as well -- it doesn't seem like a savvy move to cut over to > something that could well be a moving target until some dust settles. I understand that some people out there are concerned about the GitHub acquisition. But at this point in the maturity of the infrastructure, there are more pressing concerns than what will Microsoft do. * As seen here, https://circleci.com/gh/ghc/ghc, master is currently red on everything but x86_64-linux (sans LLVM, sans Hadrian). * This means that starting from a stock Debian Jessie, we can't get validate to pass on stock virtualized infrastrure (except for one). * So the first order of business is to get ghc HEAD to a sufficient level of quality that validate passes everywhere and so that CI becomes useful. * None of this work is GitHub specific. Nor all that CircleCI or Appveyor specific for that matter (work is currently focused on improving the test suite). * Our GitHub lock-in factor is currently low to pretty much absent, and would remain low even if the review workflow becomes more systematically GitHub centric (it already is for some small contributions). * That's because tickets remain on Trac, and the code along with the entirety of its history remains in a standard Git repository, GitHub or not. Also because GitHub is not a CI provider, those providers we do use integrate with other code hosting solutions (e.g. Appveyor with GitLab), and the surface area of CI provider-specific code is small. Now, this isn't to say that other options (e.g. GitLab or Bitbucket, for code review and/or for code hosting and/or for CI) aren't worth considering medium term. It's that I see no reason to stall first getting to a stable situation where CI is green on all "Tier 1" platforms on all types of hardware, nor to fear accepting even more contributions from GitHub users. Best, From gershomb at gmail.com Wed Jun 6 08:28:00 2018 From: gershomb at gmail.com (Gershom B) Date: Wed, 6 Jun 2018 01:28:00 -0700 Subject: [GHC DevOps Group] State of CI In-Reply-To: References: <87zi09gpr6.fsf@smart-cactus.org> Message-ID: On June 6, 2018 at 4:21:35 AM, Boespflug, Mathieu (m at tweag.io) wrote: I understand that some people out there are concerned about the GitHub acquisition. But at this point in the maturity of the infrastructure, there are more pressing concerns than what will Microsoft do. * As seen here, https://circleci.com/gh/ghc/ghc, master is currently red on everything but x86_64-linux (sans LLVM, sans Hadrian). * This means that starting from a stock Debian Jessie, we can't get validate to pass on stock virtualized infrastrure (except for one). * So the first order of business is to get ghc HEAD to a sufficient level of quality that validate passes everywhere and so that CI becomes useful. * None of this work is GitHub specific. Nor all that CircleCI or Appveyor specific for that matter (work is currently focused on improving the test suite). Right. This all sounds like good work, and a good priority to focus on, and doesn’t seem to involve needing to make any short-term decisions about changing the patch and review workflow. So it sounds like we have our hands full already and should just let things settle for the time being, and proceed as Ben has suggested :-) Cheers, Gershom -------------- next part -------------- An HTML attachment was scrubbed... URL: From andres at well-typed.com Wed Jun 6 08:34:45 2018 From: andres at well-typed.com (Andres =?utf-8?Q?L=C3=B6h?=) Date: Wed, 6 Jun 2018 09:34:45 +0100 Subject: [GHC DevOps Group] State of CI In-Reply-To: References: <87zi09gpr6.fsf@smart-cactus.org> Message-ID: <20180606083445.to7b7q45zluymior@nullzig.kosmikus.org> Hi everyone. I'm not sure if the Microsoft acquisition of github is really all that relevant. Time will show if it has any impact (positive or negative or whatever). In any case, it should probably not be a reason for us / GHC to make any decision on using Github more or less based on that right now. Given that, it seems the arguments for/against moving everything (or at least more of the development process) to Github remain largely as they have been before. The Phabricator / Phacility change of business model was news to me, but after talking to Ben and Austin a bit more, it also seems like there's no short-term detrimental effect to the change. Also, it seems the change has actually already occurred quite some time ago, and hasn't signficantly impacted GHC development up until now. So while we should perhaps monitor this, it seems the arguments for/against Phabricator are also mostly as before. I haven't been following all the discussions in as much detail as I perhaps should have, but is there currently anything that prevents us from reaching our CI goals while continuing to use Phabricator? If not, I suggest we continue trying that (which in my perception used to be the plan; if not, please correct me). Cheers, Andres On Wed, Jun 06, 2018 at 10:21:34AM +0200, Boespflug, Mathieu wrote: > Hi Gershom, > > On 6 June 2018 at 03:23, Gershom B wrote: > > > > On Tue, Jun 5, 2018 at 4:51 PM, Simon Peyton Jones > > wrote: > > > I have no inside knowledge, but I suspect that Microsoft's acquisition of Github means > > > - that GitHub will be largely undisturbed culturally > > > - that GitHub will have more oomph behind it, so it'll become yet > > > more the de-facto choice than it already is > > > > For what it's worth, the latter is definitely not happening. [...] > > Some metrics have shown that there has been a > > marked increase in imports of github accounts to gitlab in the past > > day or so. > > I assume you're referring to this: > https://twitter.com/gitlab/status/1004143715844124673. I think it's > premature to draw any inference from a single metric, let alone > "definite" ones. Adoption is the result of the *net* inflow of users > (new users minus lost users) integrated over time. With any > substantial change, there are always going to be some disgruntled > users, and others hedging their bets. That says little about the > influx (or lack thereof) of new users and about the longer term trend. > > > Certainly the expectation is that microsoft will evolve github in > > _some_ direction under its management, and this uncertainty enough is > > sufficient to drive some migration -- and should give pause to GHC > > plans as well -- it doesn't seem like a savvy move to cut over to > > something that could well be a moving target until some dust settles. > > I understand that some people out there are concerned about the GitHub > acquisition. But at this point in the maturity of the infrastructure, > there are more pressing concerns than what will Microsoft do. > > * As seen here, https://circleci.com/gh/ghc/ghc, master is currently > red on everything but x86_64-linux (sans LLVM, sans Hadrian). > * This means that starting from a stock Debian Jessie, we can't get > validate to pass on stock virtualized infrastrure (except for one). > * So the first order of business is to get ghc HEAD to a sufficient > level of quality that validate passes everywhere and so that CI > becomes useful. > * None of this work is GitHub specific. Nor all that CircleCI or > Appveyor specific for that matter (work is currently focused on > improving the test suite). > * Our GitHub lock-in factor is currently low to pretty much absent, > and would remain low even if the review workflow becomes more > systematically GitHub centric (it already is for some small > contributions). > * That's because tickets remain on Trac, and the code along with the > entirety of its history remains in a standard Git repository, GitHub > or not. Also because GitHub is not a CI provider, those providers we > do use integrate with other code hosting solutions (e.g. Appveyor with > GitLab), and the surface area of CI provider-specific code is small. > > Now, this isn't to say that other options (e.g. GitLab or Bitbucket, > for code review and/or for code hosting and/or for CI) aren't worth > considering medium term. It's that I see no reason to stall first > getting to a stable situation where CI is green on all "Tier 1" > platforms on all types of hardware, nor to fear accepting even more > contributions from GitHub users. > > Best, > _______________________________________________ > Ghc-devops-group mailing list > Ghc-devops-group at haskell.org > https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devops-group -- Andres Löh, Haskell Consultant Well-Typed LLP, http://www.well-typed.com Registered in England & Wales, OC335890 118 Wymering Mansions, Wymering Road, London W9 2NF, England From hvriedel at gmail.com Wed Jun 6 08:35:45 2018 From: hvriedel at gmail.com (Herbert Valerio Riedel) Date: Wed, 6 Jun 2018 10:35:45 +0200 Subject: [GHC DevOps Group] Phacility -> Reviewable [was: Re: State of CI] In-Reply-To: <34A477A7-428C-4B41-B6C9-7F9C47C44CDA@tweag.io> References: <87zi09gpr6.fsf@smart-cactus.org> <34A477A7-428C-4B41-B6C9-7F9C47C44CDA@tweag.io> Message-ID: On Wed, Jun 6, 2018 at 10:18 AM Manuel M T Chakravarty wrote: > Overall, the future of Phacility seems much more unpredictable to me than the future of GitHub. After all, Phacility just literally demonstrated that they are happy to change the terms mid-flight. Hence, it appears prudent to move away from Phacility (which is all I am proposing). The existing code of Phabricator is open sourced under an Apache licence afaik; so the big difference here is that if Phacility pulled the nuclear option by changing their mind and going closed-source wouldn't affect the code they already open-sourced; whereas if GitHub decides to change policies or shutdown services, you are instantly affected. Hence, it appears even more prudent to *short-term* refrain from moving more towards GitHub at this point, and possibly start looking for other options mid-term. From marlowsd at gmail.com Wed Jun 6 09:11:07 2018 From: marlowsd at gmail.com (Simon Marlow) Date: Wed, 6 Jun 2018 10:11:07 +0100 Subject: [GHC DevOps Group] State of CI In-Reply-To: References: <87zi09gpr6.fsf@smart-cactus.org> Message-ID: On 6 June 2018 at 09:21, Boespflug, Mathieu wrote: > * As seen here, https://circleci.com/gh/ghc/ghc, master is currently > red on everything but x86_64-linux (sans LLVM, sans Hadrian). > * This means that starting from a stock Debian Jessie, we can't get > validate to pass on stock virtualized infrastrure (except for one). > * So the first order of business is to get ghc HEAD to a sufficient > level of quality that validate passes everywhere and so that CI > becomes useful. > Exactly. Getting the tree green and keeping it green should definitely be the highest priority. I should point out for those who weren't aware, however, that CircleCI raised the bar for "green" from what we had previously been using, because the CircleCI build adds profiling and runs tests in a lot more ways than a plain validate does. It's taken quite a while to catch up and clean up the broken things that were preventing the CircleCI build from being clean. (of course this is all work that we would want to do anyway, I'm just mentioning it by way of explanation for why the CircleCI builds weren't clean earlier and are still not clean on many platforms) > * None of this work is GitHub specific. Nor all that CircleCI or > Appveyor specific for that matter (work is currently focused on > improving the test suite). > * Our GitHub lock-in factor is currently low to pretty much absent, > and would remain low even if the review workflow becomes more > systematically GitHub centric (it already is for some small > contributions). > * That's because tickets remain on Trac, and the code along with the > entirety of its history remains in a standard Git repository, GitHub > or not. Also because GitHub is not a CI provider, those providers we > do use integrate with other code hosting solutions (e.g. Appveyor with > GitLab), and the surface area of CI provider-specific code is small. > We should keep in mind, though, is that past code reviews is valuable content that we can't discard, nor can we easily migrate it to a different code review platform. At this point we have nearly 5K diffs on Phabricator, many of which have non-trivial code-review trails, and these are cross-referenced from Trac, emails, and other places. Even if we moved to github, we would want to keep Phabricator running so that we have access to this content, and people will experience friction though havng to deal with another system. To me, the friction caused by the transition and the inability to do a clean move is more worrying than the missing code review functionality on github. Cheers Simon > Now, this isn't to say that other options (e.g. GitLab or Bitbucket, > for code review and/or for code hosting and/or for CI) aren't worth > considering medium term. It's that I see no reason to stall first > getting to a stable situation where CI is green on all "Tier 1" > platforms on all types of hardware, nor to fear accepting even more > contributions from GitHub users. > > Best, > _______________________________________________ > Ghc-devops-group mailing list > Ghc-devops-group at haskell.org > https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devops-group > -------------- next part -------------- An HTML attachment was scrubbed... URL: From m at tweag.io Wed Jun 6 10:57:25 2018 From: m at tweag.io (Boespflug, Mathieu) Date: Wed, 6 Jun 2018 12:57:25 +0200 Subject: [GHC DevOps Group] State of CI In-Reply-To: References: <87zi09gpr6.fsf@smart-cactus.org> Message-ID: Hi Simon, On 6 June 2018 at 11:11, Simon Marlow wrote: > On 6 June 2018 at 09:21, Boespflug, Mathieu wrote: > > [...] > >> * Our GitHub lock-in factor is currently low to pretty much absent, >> and would remain low even if the review workflow becomes more >> systematically GitHub centric (it already is for some small >> contributions). >> * That's because tickets remain on Trac, and the code along with the >> entirety of its history remains in a standard Git repository, GitHub >> or not. Also because GitHub is not a CI provider, those providers we >> do use integrate with other code hosting solutions (e.g. Appveyor with >> GitLab), and the surface area of CI provider-specific code is small. > > > We should keep in mind, though, is that past code reviews is valuable > content that we can't discard, nor can we easily migrate it to a different > code review platform. At this point we have nearly 5K diffs on Phabricator, > many of which have non-trivial code-review trails, and these are > cross-referenced from Trac, emails, and other places. Indeed. And this is AFAICS the only main "lock-in factor", in that reviews on GitHub pull requests is the only "GitHub native" state. All other long term state (code, code history, tickets, code reviews) is stored elsewhere. We already use GitHub for some code reviews. If we moved elsewhere exclusively someday, then we'd mark the github.com/ghc/ghc repo as "archived". This would make it a read-only copy available forever in the future, meaning that all existing links in the GHC codebase or elsewhere would continue to work. New code reviews would happen elsewhere, no history would be lost and cross referencing would work as it did before. So if we moved e.g. from Phabricator to GitHub exclusively, then we'd do the same thing. Retain Phabricator as-is, just don't add new code reviews to it. Not that this changes anything to the plan of record regarding CI. Topic for a separate thread I guess. From hvriedel at gmail.com Wed Jun 6 11:05:49 2018 From: hvriedel at gmail.com (Herbert Valerio Riedel) Date: Wed, 6 Jun 2018 13:05:49 +0200 Subject: [GHC DevOps Group] State of CI In-Reply-To: References: <87zi09gpr6.fsf@smart-cactus.org> Message-ID: On Wed, Jun 6, 2018 at 12:57 PM Boespflug, Mathieu wrote: > then we'd mark the github.com/ghc/ghc repo as > "archived". This would make it a read-only copy available forever in > the future, meaning that all existing links in the GHC codebase or > elsewhere would continue to work. I don't think you can genuinely make such strong claims regarding "available forever" over services you don't have any control over. Only when you self-host services and thus have actual control over the domain name and provided service you can make reasonable claims about future availability. This is one of the reasons why GHC HQ in the past has preferred to self-host services to retain control for various reasons, and I remain convinced that self-hosting is and remains our best option. From manuel.chakravarty at tweag.io Wed Jun 6 11:37:19 2018 From: manuel.chakravarty at tweag.io (Manuel M T Chakravarty) Date: Wed, 6 Jun 2018 21:37:19 +1000 Subject: [GHC DevOps Group] State of CI In-Reply-To: <20180606083445.to7b7q45zluymior@nullzig.kosmikus.org> References: <87zi09gpr6.fsf@smart-cactus.org> <20180606083445.to7b7q45zluymior@nullzig.kosmikus.org> Message-ID: Hi Andres, > Am 06.06.2018 um 18:34 schrieb Andres Löh : > [..] > I haven't been following all the discussions in as much detail as I > perhaps should have, but is there currently anything that prevents us > from reaching our CI goals while continuing to use Phabricator? If not, > I suggest we continue trying that (which in my perception used to be > the plan; if not, please correct me). Yes, there is a serious obstacle to reaching our CI goals: with Phabricator we cannot currently run CI on differentials (aka pull requests), whereas with GitHub we get that functionality for free. This is why Ben wrote in his answer to my original message, > All of this certainly complicates the CI story. On one hand, I've been > a tad reluctant to spend too much time hacking Phabricator/CircleCI > integration together given the Phabricator situation. On the other hand, > CircleCI and Appveyor both only support GitHub, so a move to, for > instance, GitLab doesn’t really unblock us. We have been dancing around this issue for months now. (We can, of course, continue doing this for a few more months…) Cheers, Manuel > Cheers, > Andres > > On Wed, Jun 06, 2018 at 10:21:34AM +0200, Boespflug, Mathieu wrote: >> Hi Gershom, >> >> On 6 June 2018 at 03:23, Gershom B wrote: >>> >>> On Tue, Jun 5, 2018 at 4:51 PM, Simon Peyton Jones >>> wrote: >>>> I have no inside knowledge, but I suspect that Microsoft's acquisition of Github means >>>> - that GitHub will be largely undisturbed culturally >>>> - that GitHub will have more oomph behind it, so it'll become yet >>>> more the de-facto choice than it already is >>> >>> For what it's worth, the latter is definitely not happening. [...] >>> Some metrics have shown that there has been a >>> marked increase in imports of github accounts to gitlab in the past >>> day or so. >> >> I assume you're referring to this: >> https://twitter.com/gitlab/status/1004143715844124673. I think it's >> premature to draw any inference from a single metric, let alone >> "definite" ones. Adoption is the result of the *net* inflow of users >> (new users minus lost users) integrated over time. With any >> substantial change, there are always going to be some disgruntled >> users, and others hedging their bets. That says little about the >> influx (or lack thereof) of new users and about the longer term trend. >> >>> Certainly the expectation is that microsoft will evolve github in >>> _some_ direction under its management, and this uncertainty enough is >>> sufficient to drive some migration -- and should give pause to GHC >>> plans as well -- it doesn't seem like a savvy move to cut over to >>> something that could well be a moving target until some dust settles. >> >> I understand that some people out there are concerned about the GitHub >> acquisition. But at this point in the maturity of the infrastructure, >> there are more pressing concerns than what will Microsoft do. >> >> * As seen here, https://circleci.com/gh/ghc/ghc, master is currently >> red on everything but x86_64-linux (sans LLVM, sans Hadrian). >> * This means that starting from a stock Debian Jessie, we can't get >> validate to pass on stock virtualized infrastrure (except for one). >> * So the first order of business is to get ghc HEAD to a sufficient >> level of quality that validate passes everywhere and so that CI >> becomes useful. >> * None of this work is GitHub specific. Nor all that CircleCI or >> Appveyor specific for that matter (work is currently focused on >> improving the test suite). >> * Our GitHub lock-in factor is currently low to pretty much absent, >> and would remain low even if the review workflow becomes more >> systematically GitHub centric (it already is for some small >> contributions). >> * That's because tickets remain on Trac, and the code along with the >> entirety of its history remains in a standard Git repository, GitHub >> or not. Also because GitHub is not a CI provider, those providers we >> do use integrate with other code hosting solutions (e.g. Appveyor with >> GitLab), and the surface area of CI provider-specific code is small. >> >> Now, this isn't to say that other options (e.g. GitLab or Bitbucket, >> for code review and/or for code hosting and/or for CI) aren't worth >> considering medium term. It's that I see no reason to stall first >> getting to a stable situation where CI is green on all "Tier 1" >> platforms on all types of hardware, nor to fear accepting even more >> contributions from GitHub users. >> >> Best, >> _______________________________________________ >> Ghc-devops-group mailing list >> Ghc-devops-group at haskell.org >> https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devops-group > > -- > Andres Löh, Haskell Consultant > Well-Typed LLP, http://www.well-typed.com > > Registered in England & Wales, OC335890 > 118 Wymering Mansions, Wymering Road, London W9 2NF, England > _______________________________________________ > Ghc-devops-group mailing list > Ghc-devops-group at haskell.org > https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devops-group -------------- next part -------------- An HTML attachment was scrubbed... URL: From manuel.chakravarty at tweag.io Wed Jun 6 11:49:32 2018 From: manuel.chakravarty at tweag.io (Manuel M T Chakravarty) Date: Wed, 6 Jun 2018 21:49:32 +1000 Subject: [GHC DevOps Group] State of CI In-Reply-To: References: <87zi09gpr6.fsf@smart-cactus.org> Message-ID: <64C0CA63-B54F-4973-A8A2-044106B74500@tweag.io> > Am 06.06.2018 um 19:11 schrieb Simon Marlow : > > * None of this work is GitHub specific. Nor all that CircleCI or > Appveyor specific for that matter (work is currently focused on > improving the test suite). > * Our GitHub lock-in factor is currently low to pretty much absent, > and would remain low even if the review workflow becomes more > systematically GitHub centric (it already is for some small > contributions). > * That's because tickets remain on Trac, and the code along with the > entirety of its history remains in a standard Git repository, GitHub > or not. Also because GitHub is not a CI provider, those providers we > do use integrate with other code hosting solutions (e.g. Appveyor with > GitLab), and the surface area of CI provider-specific code is small. > > We should keep in mind, though, is that past code reviews is valuable content that we can't discard, nor can we easily migrate it to a different code review platform. At this point we have nearly 5K diffs on Phabricator, many of which have non-trivial code-review trails, and these are cross-referenced from Trac, emails, and other places. Even if we moved to github, we would want to keep Phabricator running so that we have access to this content, and people will experience friction though havng to deal with another system. > > To me, the friction caused by the transition and the inability to do a clean move is more worrying than the missing code review functionality on github. Actually, to me this is a red flag. Core reviews shouldn’t be essential documentation. I wonder what has been going wrong so that we have got to this situation. If important points are uncovered or documented in code reviews, that information should be included in either the source code or associated documentation. (Having git history as an important record for the evolution of the code is fine as it is now standard to not only have one snapshot of the source, but a source control history of the source as the definitive record of a piece of software. And as we painfully discover, much in contrast to code reviews, the source control history is recorded in a widely understood standard format.) Cheers, Manuel -------------- next part -------------- An HTML attachment was scrubbed... URL: From simonpj at microsoft.com Wed Jun 6 11:58:12 2018 From: simonpj at microsoft.com (Simon Peyton Jones) Date: Wed, 6 Jun 2018 11:58:12 +0000 Subject: [GHC DevOps Group] State of CI In-Reply-To: <64C0CA63-B54F-4973-A8A2-044106B74500@tweag.io> References: <87zi09gpr6.fsf@smart-cactus.org> <64C0CA63-B54F-4973-A8A2-044106B74500@tweag.io> Message-ID: We should keep in mind, though, is that past code reviews is valuable content that we can't discard, Like Manuel I’m not at all sure about this. I *do* regard the Trac conversation as a long-term asset, and often refer to tickets from Notes, as a way to say “here’s a more extensive discussion of what’s in the Note”. But the Phab discussion of “please refactor this or that” seems far less valuable. And actually I don’t think it is substantially cross-referenced from elsewhere. Where there is substantive conversation about the approach, I’d rather see that on Trac. So for me, long term access to the code-review trail is not very important Simon From: Ghc-devops-group On Behalf Of Manuel M T Chakravarty Sent: 06 June 2018 12:50 To: Simon Marlow Cc: ghc-devops-group at haskell.org Subject: Re: [GHC DevOps Group] State of CI Am 06.06.2018 um 19:11 schrieb Simon Marlow >: * None of this work is GitHub specific. Nor all that CircleCI or Appveyor specific for that matter (work is currently focused on improving the test suite). * Our GitHub lock-in factor is currently low to pretty much absent, and would remain low even if the review workflow becomes more systematically GitHub centric (it already is for some small contributions). * That's because tickets remain on Trac, and the code along with the entirety of its history remains in a standard Git repository, GitHub or not. Also because GitHub is not a CI provider, those providers we do use integrate with other code hosting solutions (e.g. Appveyor with GitLab), and the surface area of CI provider-specific code is small. We should keep in mind, though, is that past code reviews is valuable content that we can't discard, nor can we easily migrate it to a different code review platform. At this point we have nearly 5K diffs on Phabricator, many of which have non-trivial code-review trails, and these are cross-referenced from Trac, emails, and other places. Even if we moved to github, we would want to keep Phabricator running so that we have access to this content, and people will experience friction though havng to deal with another system. To me, the friction caused by the transition and the inability to do a clean move is more worrying than the missing code review functionality on github. Actually, to me this is a red flag. Core reviews shouldn’t be essential documentation. I wonder what has been going wrong so that we have got to this situation. If important points are uncovered or documented in code reviews, that information should be included in either the source code or associated documentation. (Having git history as an important record for the evolution of the code is fine as it is now standard to not only have one snapshot of the source, but a source control history of the source as the definitive record of a piece of software. And as we painfully discover, much in contrast to code reviews, the source control history is recorded in a widely understood standard format.) Cheers, Manuel -------------- next part -------------- An HTML attachment was scrubbed... URL: From marlowsd at gmail.com Wed Jun 6 12:08:54 2018 From: marlowsd at gmail.com (Simon Marlow) Date: Wed, 6 Jun 2018 13:08:54 +0100 Subject: [GHC DevOps Group] State of CI In-Reply-To: <64C0CA63-B54F-4973-A8A2-044106B74500@tweag.io> References: <87zi09gpr6.fsf@smart-cactus.org> <64C0CA63-B54F-4973-A8A2-044106B74500@tweag.io> Message-ID: On 6 June 2018 at 12:49, Manuel M T Chakravarty wrote: > Am 06.06.2018 um 19:11 schrieb Simon Marlow : > > * None of this work is GitHub specific. Nor all that CircleCI or >> Appveyor specific for that matter (work is currently focused on >> improving the test suite). >> * Our GitHub lock-in factor is currently low to pretty much absent, >> and would remain low even if the review workflow becomes more >> systematically GitHub centric (it already is for some small >> contributions). >> * That's because tickets remain on Trac, and the code along with the >> entirety of its history remains in a standard Git repository, GitHub >> or not. Also because GitHub is not a CI provider, those providers we >> do use integrate with other code hosting solutions (e.g. Appveyor with >> GitLab), and the surface area of CI provider-specific code is small. >> > > We should keep in mind, though, is that past code reviews is valuable > content that we can't discard, nor can we easily migrate it to a different > code review platform. At this point we have nearly 5K diffs on Phabricator, > many of which have non-trivial code-review trails, and these are > cross-referenced from Trac, emails, and other places. Even if we moved to > github, we would want to keep Phabricator running so that we have access to > this content, and people will experience friction though havng to deal with > another system. > > To me, the friction caused by the transition and the inability to do a > clean move is more worrying than the missing code review functionality on > github. > > > Actually, to me this is a red flag. Core reviews shouldn’t be essential > documentation. I wonder what has been going wrong so that we have got to > this situation. > > If important points are uncovered or documented in code reviews, that > information should be included in either the source code or associated > documentation. (Having git history as an important record for the evolution > of the code is fine as it is now standard to not only have one snapshot of > the source, but a source control history of the source as the definitive > record of a piece of software. And as we painfully discover, much in > contrast to code reviews, the source control history is recorded in a > widely understood standard format.) > In an ideal world all the history you need is in the repository, but in practice often the extra context of the code review is useful in understanding how things got to be the way they are. We could require people to be meticulous about trying to anticipate which parts of the discussion will be useful and copying those into the commit log, but in practice this isn't going to happen. Code reviews - just like mailing list archives and the discussion on old tickets - is useful history and we need to keep it. Cheers Simon > > Cheers, > Manuel > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From moritz.angermann at gmail.com Wed Jun 6 12:14:35 2018 From: moritz.angermann at gmail.com (Moritz Angermann) Date: Wed, 6 Jun 2018 20:14:35 +0800 Subject: [GHC DevOps Group] State of CI In-Reply-To: References: <87zi09gpr6.fsf@smart-cactus.org> <64C0CA63-B54F-4973-A8A2-044106B74500@tweag.io> Message-ID: While we are discussing code review, I found what jane street demoed here quite interesting: https://www.youtube.com/watch?v=MUqvXHEjmus From the Code Reviews I've done/seen in Phabricator, I'm inclined to say that I've rarely went back to look through the review history. Unless it was one of my Diffs, and I had forgotten what it was all about in the mean time. Cheers, Moritz > On Jun 6, 2018, at 8:08 PM, Simon Marlow wrote: > > On 6 June 2018 at 12:49, Manuel M T Chakravarty wrote: >> Am 06.06.2018 um 19:11 schrieb Simon Marlow : >> >> * None of this work is GitHub specific. Nor all that CircleCI or >> Appveyor specific for that matter (work is currently focused on >> improving the test suite). >> * Our GitHub lock-in factor is currently low to pretty much absent, >> and would remain low even if the review workflow becomes more >> systematically GitHub centric (it already is for some small >> contributions). >> * That's because tickets remain on Trac, and the code along with the >> entirety of its history remains in a standard Git repository, GitHub >> or not. Also because GitHub is not a CI provider, those providers we >> do use integrate with other code hosting solutions (e.g. Appveyor with >> GitLab), and the surface area of CI provider-specific code is small. >> >> We should keep in mind, though, is that past code reviews is valuable content that we can't discard, nor can we easily migrate it to a different code review platform. At this point we have nearly 5K diffs on Phabricator, many of which have non-trivial code-review trails, and these are cross-referenced from Trac, emails, and other places. Even if we moved to github, we would want to keep Phabricator running so that we have access to this content, and people will experience friction though havng to deal with another system. >> >> To me, the friction caused by the transition and the inability to do a clean move is more worrying than the missing code review functionality on github. > > Actually, to me this is a red flag. Core reviews shouldn’t be essential documentation. I wonder what has been going wrong so that we have got to this situation. > > If important points are uncovered or documented in code reviews, that information should be included in either the source code or associated documentation. (Having git history as an important record for the evolution of the code is fine as it is now standard to not only have one snapshot of the source, but a source control history of the source as the definitive record of a piece of software. And as we painfully discover, much in contrast to code reviews, the source control history is recorded in a widely understood standard format.) > > In an ideal world all the history you need is in the repository, but in practice often the extra context of the code review is useful in understanding how things got to be the way they are. We could require people to be meticulous about trying to anticipate which parts of the discussion will be useful and copying those into the commit log, but in practice this isn't going to happen. Code reviews - just like mailing list archives and the discussion on old tickets - is useful history and we need to keep it. > > Cheers > Simon > > > > Cheers, > Manuel > > > _______________________________________________ > Ghc-devops-group mailing list > Ghc-devops-group at haskell.org > https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devops-group -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 488 bytes Desc: Message signed with OpenPGP URL: From marlowsd at gmail.com Wed Jun 6 12:44:26 2018 From: marlowsd at gmail.com (Simon Marlow) Date: Wed, 6 Jun 2018 13:44:26 +0100 Subject: [GHC DevOps Group] State of CI In-Reply-To: References: <87zi09gpr6.fsf@smart-cactus.org> <64C0CA63-B54F-4973-A8A2-044106B74500@tweag.io> Message-ID: I think this reflects a different philosophy about code review. If we say that code-review is only for small-scale suggestions about the actual code changes, then yes I'd agree it's not all that useful to keep that history around. But we can (and sometimes do) have high-level architectural discussions as part of code review too, and indeed I think it's arguably the right thing to have these discussion around concrete code proposals. We keep all the discussion of the code in one place, and the discussion is attached to the evolving code patches. In this world, the code discussions really are valuable history. Interestingly, I think this discussion has really teased out a key difference - perhaps the reason people prefer different tools here is because they're starting from different perspectives about what the tools are for? I don't think there's one true way here. Personally I've been exposed to multiple different working styles, especially having one foot in industry and one in the open-source world, and I've seen different philosophies that work. We could as a project decide that we don't want to go the way of having high-level discussion alongside code review, in which case that's OK (I would slightly prefer to move in the other direction, but that's just my preference). Cheers Simon On 6 June 2018 at 12:58, Simon Peyton Jones wrote: > We should keep in mind, though, is that past code reviews is valuable > content that we can't discard, > > > > Like Manuel I’m not at all sure about this. > > > > I **do** regard the Trac conversation as a long-term asset, and often > refer to tickets from Notes, as a way to say “here’s a more extensive > discussion of what’s in the Note”. > > > > But the Phab discussion of “please refactor this or that” seems far less > valuable. And actually I don’t think it is substantially cross-referenced > from elsewhere. Where there is substantive conversation about the > approach, I’d rather see that on Trac. > > > > So for me, long term access to the code-review trail is not very important > > > > Simon > > > > *From:* Ghc-devops-group *On > Behalf Of *Manuel M T Chakravarty > *Sent:* 06 June 2018 12:50 > *To:* Simon Marlow > *Cc:* ghc-devops-group at haskell.org > *Subject:* Re: [GHC DevOps Group] State of CI > > > > Am 06.06.2018 um 19:11 schrieb Simon Marlow : > > > > * None of this work is GitHub specific. Nor all that CircleCI or > Appveyor specific for that matter (work is currently focused on > improving the test suite). > * Our GitHub lock-in factor is currently low to pretty much absent, > and would remain low even if the review workflow becomes more > systematically GitHub centric (it already is for some small > contributions). > * That's because tickets remain on Trac, and the code along with the > entirety of its history remains in a standard Git repository, GitHub > or not. Also because GitHub is not a CI provider, those providers we > do use integrate with other code hosting solutions (e.g. Appveyor with > GitLab), and the surface area of CI provider-specific code is small. > > > > We should keep in mind, though, is that past code reviews is valuable > content that we can't discard, nor can we easily migrate it to a different > code review platform. At this point we have nearly 5K diffs on Phabricator, > many of which have non-trivial code-review trails, and these are > cross-referenced from Trac, emails, and other places. Even if we moved to > github, we would want to keep Phabricator running so that we have access to > this content, and people will experience friction though havng to deal with > another system. > > > > To me, the friction caused by the transition and the inability to do a > clean move is more worrying than the missing code review functionality on > github. > > > > Actually, to me this is a red flag. Core reviews shouldn’t be essential > documentation. I wonder what has been going wrong so that we have got to > this situation. > > > > If important points are uncovered or documented in code reviews, that > information should be included in either the source code or associated > documentation. (Having git history as an important record for the evolution > of the code is fine as it is now standard to not only have one snapshot of > the source, but a source control history of the source as the definitive > record of a piece of software. And as we painfully discover, much in > contrast to code reviews, the source control history is recorded in a > widely understood standard format.) > > > > Cheers, > > Manuel > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From simonpj at microsoft.com Wed Jun 6 13:47:17 2018 From: simonpj at microsoft.com (Simon Peyton Jones) Date: Wed, 6 Jun 2018 13:47:17 +0000 Subject: [GHC DevOps Group] State of CI In-Reply-To: References: <87zi09gpr6.fsf@smart-cactus.org> <64C0CA63-B54F-4973-A8A2-044106B74500@tweag.io> Message-ID: Interestingly, I think this discussion has really teased out a key difference - perhaps the reason people prefer different tools here is because they're starting from different perspectives about what the tools are for? I think you are right here. Personally, I find that Phab focuses my attention on the *code*, rather than the design or architecture. Even where discussions about the latter happen, they are buried in a sea of noise about low-level suggestions. So my personal preference is to keep the higher level stuff on Trac (or even a wiki page) and regard the code review discussion as ephemeral. Simon From: Simon Marlow Sent: 06 June 2018 13:44 To: Simon Peyton Jones Cc: Manuel M T Chakravarty ; ghc-devops-group at haskell.org Subject: Re: [GHC DevOps Group] State of CI I think this reflects a different philosophy about code review. If we say that code-review is only for small-scale suggestions about the actual code changes, then yes I'd agree it's not all that useful to keep that history around. But we can (and sometimes do) have high-level architectural discussions as part of code review too, and indeed I think it's arguably the right thing to have these discussion around concrete code proposals. We keep all the discussion of the code in one place, and the discussion is attached to the evolving code patches. In this world, the code discussions really are valuable history. Interestingly, I think this discussion has really teased out a key difference - perhaps the reason people prefer different tools here is because they're starting from different perspectives about what the tools are for? I don't think there's one true way here. Personally I've been exposed to multiple different working styles, especially having one foot in industry and one in the open-source world, and I've seen different philosophies that work. We could as a project decide that we don't want to go the way of having high-level discussion alongside code review, in which case that's OK (I would slightly prefer to move in the other direction, but that's just my preference). Cheers Simon On 6 June 2018 at 12:58, Simon Peyton Jones > wrote: We should keep in mind, though, is that past code reviews is valuable content that we can't discard, Like Manuel I’m not at all sure about this. I *do* regard the Trac conversation as a long-term asset, and often refer to tickets from Notes, as a way to say “here’s a more extensive discussion of what’s in the Note”. But the Phab discussion of “please refactor this or that” seems far less valuable. And actually I don’t think it is substantially cross-referenced from elsewhere. Where there is substantive conversation about the approach, I’d rather see that on Trac. So for me, long term access to the code-review trail is not very important Simon From: Ghc-devops-group > On Behalf Of Manuel M T Chakravarty Sent: 06 June 2018 12:50 To: Simon Marlow > Cc: ghc-devops-group at haskell.org Subject: Re: [GHC DevOps Group] State of CI Am 06.06.2018 um 19:11 schrieb Simon Marlow >: * None of this work is GitHub specific. Nor all that CircleCI or Appveyor specific for that matter (work is currently focused on improving the test suite). * Our GitHub lock-in factor is currently low to pretty much absent, and would remain low even if the review workflow becomes more systematically GitHub centric (it already is for some small contributions). * That's because tickets remain on Trac, and the code along with the entirety of its history remains in a standard Git repository, GitHub or not. Also because GitHub is not a CI provider, those providers we do use integrate with other code hosting solutions (e.g. Appveyor with GitLab), and the surface area of CI provider-specific code is small. We should keep in mind, though, is that past code reviews is valuable content that we can't discard, nor can we easily migrate it to a different code review platform. At this point we have nearly 5K diffs on Phabricator, many of which have non-trivial code-review trails, and these are cross-referenced from Trac, emails, and other places. Even if we moved to github, we would want to keep Phabricator running so that we have access to this content, and people will experience friction though havng to deal with another system. To me, the friction caused by the transition and the inability to do a clean move is more worrying than the missing code review functionality on github. Actually, to me this is a red flag. Core reviews shouldn’t be essential documentation. I wonder what has been going wrong so that we have got to this situation. If important points are uncovered or documented in code reviews, that information should be included in either the source code or associated documentation. (Having git history as an important record for the evolution of the code is fine as it is now standard to not only have one snapshot of the source, but a source control history of the source as the definitive record of a piece of software. And as we painfully discover, much in contrast to code reviews, the source control history is recorded in a widely understood standard format.) Cheers, Manuel -------------- next part -------------- An HTML attachment was scrubbed... URL: From manuel.chakravarty at tweag.io Thu Jun 7 06:39:38 2018 From: manuel.chakravarty at tweag.io (Manuel M T Chakravarty) Date: Thu, 7 Jun 2018 16:39:38 +1000 Subject: [GHC DevOps Group] State of CI In-Reply-To: References: <87zi09gpr6.fsf@smart-cactus.org> <64C0CA63-B54F-4973-A8A2-044106B74500@tweag.io> Message-ID: I very much feel like SimonPJ about this. Moreover, I think, there are two important technical reasons to prefer this style. Firstly, I don’t want to be dependent on any one code review tool. The lock-in that Phabricator has been able to create here is quite tiresome (even if Phacility is probably quite happy about the situation). Secondly, I’d be willing to bet that the history in code reviews is useful only to very few people, namely those who participated in the respective code reviews. This is in direct opposition to creating an open project, which tries to level the playing field as far as possible. Hence, IMHO it runs counter to the aims of this group. And, SimonM, as you bring up the differences between open-source and closed source. I agree that this would be less of an issue in a closed source project. However, I always thought, we want to make contributing to GHC as easy as possible. Cheers, Manuel > Am 06.06.2018 um 23:47 schrieb Simon Peyton Jones via Ghc-devops-group : > > Interestingly, I think this discussion has really teased out a key difference - perhaps the reason people prefer different tools here is because they're starting from different perspectives about what the tools are for? > > I think you are right here. Personally, I find that Phab focuses my attention on the *code*, rather than the design or architecture. Even where discussions about the latter happen, they are buried in a sea of noise about low-level suggestions. So my personal preference is to keep the higher level stuff on Trac (or even a wiki page) and regard the code review discussion as ephemeral. > > Simon > > From: Simon Marlow > Sent: 06 June 2018 13:44 > To: Simon Peyton Jones > Cc: Manuel M T Chakravarty ; ghc-devops-group at haskell.org > Subject: Re: [GHC DevOps Group] State of CI > > I think this reflects a different philosophy about code review. If we say that code-review is only for small-scale suggestions about the actual code changes, then yes I'd agree it's not all that useful to keep that history around. But we can (and sometimes do) have high-level architectural discussions as part of code review too, and indeed I think it's arguably the right thing to have these discussion around concrete code proposals. We keep all the discussion of the code in one place, and the discussion is attached to the evolving code patches. In this world, the code discussions really are valuable history. > > > > Interestingly, I think this discussion has really teased out a key difference - perhaps the reason people prefer different tools here is because they're starting from different perspectives about what the tools are for? I don't think there's one true way here. Personally I've been exposed to multiple different working styles, especially having one foot in industry and one in the open-source world, and I've seen different philosophies that work. We could as a project decide that we don't want to go the way of having high-level discussion alongside code review, in which case that's OK (I would slightly prefer to move in the other direction, but that's just my preference). > > > > Cheers > > Simon > > > > > > On 6 June 2018 at 12:58, Simon Peyton Jones > wrote: > > We should keep in mind, though, is that past code reviews is valuable content that we can't discard, > > Like Manuel I’m not at all sure about this. > > I *do* regard the Trac conversation as a long-term asset, and often refer to tickets from Notes, as a way to say “here’s a more extensive discussion of what’s in the Note”. > > But the Phab discussion of “please refactor this or that” seems far less valuable. And actually I don’t think it is substantially cross-referenced from elsewhere. Where there is substantive conversation about the approach, I’d rather see that on Trac. > > So for me, long term access to the code-review trail is not very important > > Simon > > From: Ghc-devops-group > On Behalf Of Manuel M T Chakravarty > Sent: 06 June 2018 12:50 > To: Simon Marlow > > Cc: ghc-devops-group at haskell.org > Subject: Re: [GHC DevOps Group] State of CI > > Am 06.06.2018 um 19:11 schrieb Simon Marlow >: > > * None of this work is GitHub specific. Nor all that CircleCI or > Appveyor specific for that matter (work is currently focused on > improving the test suite). > * Our GitHub lock-in factor is currently low to pretty much absent, > and would remain low even if the review workflow becomes more > systematically GitHub centric (it already is for some small > contributions). > * That's because tickets remain on Trac, and the code along with the > entirety of its history remains in a standard Git repository, GitHub > or not. Also because GitHub is not a CI provider, those providers we > do use integrate with other code hosting solutions (e.g. Appveyor with > GitLab), and the surface area of CI provider-specific code is small. > > We should keep in mind, though, is that past code reviews is valuable content that we can't discard, nor can we easily migrate it to a different code review platform. At this point we have nearly 5K diffs on Phabricator, many of which have non-trivial code-review trails, and these are cross-referenced from Trac, emails, and other places. Even if we moved to github, we would want to keep Phabricator running so that we have access to this content, and people will experience friction though havng to deal with another system. > > To me, the friction caused by the transition and the inability to do a clean move is more worrying than the missing code review functionality on github. > > Actually, to me this is a red flag. Core reviews shouldn’t be essential documentation. I wonder what has been going wrong so that we have got to this situation. > > If important points are uncovered or documented in code reviews, that information should be included in either the source code or associated documentation. (Having git history as an important record for the evolution of the code is fine as it is now standard to not only have one snapshot of the source, but a source control history of the source as the definitive record of a piece of software. And as we painfully discover, much in contrast to code reviews, the source control history is recorded in a widely understood standard format.) > > Cheers, > Manuel > > > _______________________________________________ > Ghc-devops-group mailing list > Ghc-devops-group at haskell.org > https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devops-group -------------- next part -------------- An HTML attachment was scrubbed... URL: From marlowsd at gmail.com Thu Jun 7 08:22:40 2018 From: marlowsd at gmail.com (Simon Marlow) Date: Thu, 7 Jun 2018 09:22:40 +0100 Subject: [GHC DevOps Group] State of CI In-Reply-To: References: <87zi09gpr6.fsf@smart-cactus.org> <64C0CA63-B54F-4973-A8A2-044106B74500@tweag.io> Message-ID: The first point I think is reasonable, being independent of any one code review tool would be an advantage. However, I'm prepared to sacrifice that, because I think the conclusion it leads to is suboptimal - that code review discussion is ephemeral, and we cannot rely on having access to it after the code is committed. To make this concrete, I picked out some examples of diffs I've been involved in recently. I'm sure there are plenty more examples like this: https://phabricator.haskell.org/D42 (click on "Show Older Comments" to see all the discussion) https://phabricator.haskell.org/D4327 https://phabricator.haskell.org/D4324 https://phabricator.haskell.org/D4679 https://phabricator.haskell.org/D4644 https://phabricator.haskell.org/D4722 https://phabricator.haskell.org/D4302 The discussion here is substantial and I would be really sad to lose it, and I think its relevance goes beyond just the people involved in the review. Note the cross-referencing between diffs, the links to code, the comments on particular code fragments, and links to pastes hosted by Phabricator. It's definitely a lot more than just small-scale suggestions for improvements or refactoring. We also have cross-references from Trac tickets to Phabricator diffs, and every diff committed contains a link to the code review (I use these links a lot). There are links to Phabricator from mailing lists and wiki pages. So we could stop doing this, and discuss only on Trac tickets or mailing lists. (wiki is not good for discussion). After all, that's what we used to do before code review, and we managed. But I think to do that would be a loss - GHC development has felt a lot more engaging since we added code review, and restricting it to ephemeral discussion would curtail its potential dramatically. I'd have to think twice before writing a comment on a diff - if I want to discuss something about the overall approach, I'd have to leave a link to Trac (and create a ticket if there wasn't one), which is a huge faff. Ok, so I've made my case. This is not me stamping my foot and saying "we must do it this way" by any means! If there's consensus to change things then I'll happily adjust my workflows. Cheers Simon On 7 June 2018 at 07:39, Manuel M T Chakravarty wrote: > I very much feel like SimonPJ about this. > > Moreover, I think, there are two important technical reasons to prefer > this style. Firstly, I don’t want to be dependent on any one code review > tool. The lock-in that Phabricator has been able to create here is quite > tiresome (even if Phacility is probably quite happy about the situation). > > Secondly, I’d be willing to bet that the history in code reviews is useful > only to very few people, namely those who participated in the respective > code reviews. This is in direct opposition to creating an open project, > which tries to level the playing field as far as possible. Hence, IMHO it > runs counter to the aims of this group. > > And, SimonM, as you bring up the differences between open-source and > closed source. I agree that this would be less of an issue in a closed > source project. However, I always thought, we want to make contributing to > GHC as easy as possible. > > Cheers, > Manuel > > Am 06.06.2018 um 23:47 schrieb Simon Peyton Jones via Ghc-devops-group < > ghc-devops-group at haskell.org>: > > Interestingly, I think this discussion has really teased out a key > difference - perhaps the reason people prefer different tools here is > because they're starting from different perspectives about what the tools > are for? > > I think you are right here. Personally, I find that Phab focuses my > attention on the **code**, rather than the design or architecture. Even > where discussions about the latter happen, they are buried in a sea of > noise about low-level suggestions. So my personal preference is to keep > the higher level stuff on Trac (or even a wiki page) and regard the code > review discussion as ephemeral. > > Simon > > *From:* Simon Marlow > *Sent:* 06 June 2018 13:44 > *To:* Simon Peyton Jones > *Cc:* Manuel M T Chakravarty ; > ghc-devops-group at haskell.org > *Subject:* Re: [GHC DevOps Group] State of CI > > > I think this reflects a different philosophy about code review. If we say > that code-review is only for small-scale suggestions about the actual code > changes, then yes I'd agree it's not all that useful to keep that history > around. But we can (and sometimes do) have high-level architectural > discussions as part of code review too, and indeed I think it's arguably > the right thing to have these discussion around concrete code proposals. We > keep all the discussion of the code in one place, and the discussion is > attached to the evolving code patches. In this world, the code discussions > really are valuable history. > > > > Interestingly, I think this discussion has really teased out a key > difference - perhaps the reason people prefer different tools here is > because they're starting from different perspectives about what the tools > are for? I don't think there's one true way here. Personally I've been > exposed to multiple different working styles, especially having one foot in > industry and one in the open-source world, and I've seen different > philosophies that work. We could as a project decide that we don't want to > go the way of having high-level discussion alongside code review, in which > case that's OK (I would slightly prefer to move in the other direction, but > that's just my preference). > > > > Cheers > > Simon > > > > > > On 6 June 2018 at 12:58, Simon Peyton Jones wrote: > > We should keep in mind, though, is that past code reviews is valuable > content that we can't discard, > > Like Manuel I’m not at all sure about this. > > I **do** regard the Trac conversation as a long-term asset, and often > refer to tickets from Notes, as a way to say “here’s a more extensive > discussion of what’s in the Note”. > > But the Phab discussion of “please refactor this or that” seems far less > valuable. And actually I don’t think it is substantially cross-referenced > from elsewhere. Where there is substantive conversation about the > approach, I’d rather see that on Trac. > > So for me, long term access to the code-review trail is not very important > > Simon > > *From:* Ghc-devops-group *On > Behalf Of *Manuel M T Chakravarty > *Sent:* 06 June 2018 12:50 > *To:* Simon Marlow > *Cc:* ghc-devops-group at haskell.org > *Subject:* Re: [GHC DevOps Group] State of CI > > > Am 06.06.2018 um 19:11 schrieb Simon Marlow : > > > * None of this work is GitHub specific. Nor all that CircleCI or > Appveyor specific for that matter (work is currently focused on > improving the test suite). > * Our GitHub lock-in factor is currently low to pretty much absent, > and would remain low even if the review workflow becomes more > systematically GitHub centric (it already is for some small > contributions). > * That's because tickets remain on Trac, and the code along with the > entirety of its history remains in a standard Git repository, GitHub > or not. Also because GitHub is not a CI provider, those providers we > do use integrate with other code hosting solutions (e.g. Appveyor with > GitLab), and the surface area of CI provider-specific code is small. > > > We should keep in mind, though, is that past code reviews is valuable > content that we can't discard, nor can we easily migrate it to a different > code review platform. At this point we have nearly 5K diffs on Phabricator, > many of which have non-trivial code-review trails, and these are > cross-referenced from Trac, emails, and other places. Even if we moved to > github, we would want to keep Phabricator running so that we have access to > this content, and people will experience friction though havng to deal with > another system. > > To me, the friction caused by the transition and the inability to do a > clean move is more worrying than the missing code review functionality on > github. > > > Actually, to me this is a red flag. Core reviews shouldn’t be essential > documentation. I wonder what has been going wrong so that we have got to > this situation. > > If important points are uncovered or documented in code reviews, that > information should be included in either the source code or associated > documentation. (Having git history as an important record for the evolution > of the code is fine as it is now standard to not only have one snapshot of > the source, but a source control history of the source as the definitive > record of a piece of software. And as we painfully discover, much in > contrast to code reviews, the source control history is recorded in a > widely understood standard format.) > > Cheers, > Manuel > > > > _______________________________________________ > Ghc-devops-group mailing list > Ghc-devops-group at haskell.org > https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devops-group > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hvriedel at gmail.com Thu Jun 7 09:53:49 2018 From: hvriedel at gmail.com (Herbert Valerio Riedel) Date: Thu, 7 Jun 2018 11:53:49 +0200 Subject: [GHC DevOps Group] State of CI In-Reply-To: References: <87zi09gpr6.fsf@smart-cactus.org> <64C0CA63-B54F-4973-A8A2-044106B74500@tweag.io> Message-ID: I strongly agree here with Simon. For me, the discussions in Phabricator are often very valuable add-on historical information which provide additional insights into how things evolved at a sub-commit level -- and obviously, people aren't super disciplined and so discussions that might have better belonged into Trac ended up on Phabricator instead (and I don't mind anymore -- I'm just happy to have this information accessible somewhere). And especially in open-source projects, you often make trade-offs in terms of discipline in order not to annoy everyone too much, and unless you want to put people in charge which enjoy taking care that all interesting bits of discussions have been accurately summarised in either commit messages or in-source Notes, this the reality we're gonna have. And I don't see the big deal. We didn't decide on a whim back then to start using Phabricator as this did constitute a level of investment and commitment (in fact, I had some concerns back then we'd end up spreading discussions over Trac and Phab; but since Phab & Trac are self-hosted services of ours that isssue wasn't a big deal; and fwiw, self-hosting an OSS licensed tool makes vendor-lock-in arguments sound absurd and ironical, as avoiding vendor-lock-in is exactly the reason to use OSS rather than proprietary software or cloud-based SaaS offerings...). On Thu, Jun 7, 2018 at 10:22 AM Simon Marlow wrote: > > The first point I think is reasonable, being independent of any one code review tool would be an advantage. However, I'm prepared to sacrifice that, because I think the conclusion it leads to is suboptimal - that code review discussion is ephemeral, and we cannot rely on having access to it after the code is committed. > > To make this concrete, I picked out some examples of diffs I've been involved in recently. I'm sure there are plenty more examples like this: > > https://phabricator.haskell.org/D42 (click on "Show Older Comments" to see all the discussion) > https://phabricator.haskell.org/D4327 > https://phabricator.haskell.org/D4324 > https://phabricator.haskell.org/D4679 > https://phabricator.haskell.org/D4644 > https://phabricator.haskell.org/D4722 > https://phabricator.haskell.org/D4302 > > The discussion here is substantial and I would be really sad to lose it, and I think its relevance goes beyond just the people involved in the review. Note the cross-referencing between diffs, the links to code, the comments on particular code fragments, and links to pastes hosted by Phabricator. It's definitely a lot more than just small-scale suggestions for improvements or refactoring. > > We also have cross-references from Trac tickets to Phabricator diffs, and every diff committed contains a link to the code review (I use these links a lot). There are links to Phabricator from mailing lists and wiki pages. > > So we could stop doing this, and discuss only on Trac tickets or mailing lists. (wiki is not good for discussion). After all, that's what we used to do before code review, and we managed. But I think to do that would be a loss - GHC development has felt a lot more engaging since we added code review, and restricting it to ephemeral discussion would curtail its potential dramatically. I'd have to think twice before writing a comment on a diff - if I want to discuss something about the overall approach, I'd have to leave a link to Trac (and create a ticket if there wasn't one), which is a huge faff. > > Ok, so I've made my case. This is not me stamping my foot and saying "we must do it this way" by any means! If there's consensus to change things then I'll happily adjust my workflows. > > Cheers > Simon > > > > > On 7 June 2018 at 07:39, Manuel M T Chakravarty wrote: >> >> I very much feel like SimonPJ about this. >> >> Moreover, I think, there are two important technical reasons to prefer this style. Firstly, I don’t want to be dependent on any one code review tool. The lock-in that Phabricator has been able to create here is quite tiresome (even if Phacility is probably quite happy about the situation). >> >> Secondly, I’d be willing to bet that the history in code reviews is useful only to very few people, namely those who participated in the respective code reviews. This is in direct opposition to creating an open project, which tries to level the playing field as far as possible. Hence, IMHO it runs counter to the aims of this group. >> >> And, SimonM, as you bring up the differences between open-source and closed source. I agree that this would be less of an issue in a closed source project. However, I always thought, we want to make contributing to GHC as easy as possible. >> >> Cheers, >> Manuel >> >> Am 06.06.2018 um 23:47 schrieb Simon Peyton Jones via Ghc-devops-group : >> >> Interestingly, I think this discussion has really teased out a key difference - perhaps the reason people prefer different tools here is because they're starting from different perspectives about what the tools are for? >> >> I think you are right here. Personally, I find that Phab focuses my attention on the *code*, rather than the design or architecture. Even where discussions about the latter happen, they are buried in a sea of noise about low-level suggestions. So my personal preference is to keep the higher level stuff on Trac (or even a wiki page) and regard the code review discussion as ephemeral. >> >> Simon >> >> From: Simon Marlow >> Sent: 06 June 2018 13:44 >> To: Simon Peyton Jones >> Cc: Manuel M T Chakravarty ; ghc-devops-group at haskell.org >> Subject: Re: [GHC DevOps Group] State of CI >> >> >> I think this reflects a different philosophy about code review. If we say that code-review is only for small-scale suggestions about the actual code changes, then yes I'd agree it's not all that useful to keep that history around. But we can (and sometimes do) have high-level architectural discussions as part of code review too, and indeed I think it's arguably the right thing to have these discussion around concrete code proposals. We keep all the discussion of the code in one place, and the discussion is attached to the evolving code patches. In this world, the code discussions really are valuable history. >> >> >> >> Interestingly, I think this discussion has really teased out a key difference - perhaps the reason people prefer different tools here is because they're starting from different perspectives about what the tools are for? I don't think there's one true way here. Personally I've been exposed to multiple different working styles, especially having one foot in industry and one in the open-source world, and I've seen different philosophies that work. We could as a project decide that we don't want to go the way of having high-level discussion alongside code review, in which case that's OK (I would slightly prefer to move in the other direction, but that's just my preference). >> >> >> >> Cheers >> >> Simon >> >> >> >> >> >> On 6 June 2018 at 12:58, Simon Peyton Jones wrote: >> >> We should keep in mind, though, is that past code reviews is valuable content that we can't discard, >> >> Like Manuel I’m not at all sure about this. >> >> I *do* regard the Trac conversation as a long-term asset, and often refer to tickets from Notes, as a way to say “here’s a more extensive discussion of what’s in the Note”. >> >> But the Phab discussion of “please refactor this or that” seems far less valuable. And actually I don’t think it is substantially cross-referenced from elsewhere. Where there is substantive conversation about the approach, I’d rather see that on Trac. >> >> So for me, long term access to the code-review trail is not very important >> >> Simon >> >> From: Ghc-devops-group On Behalf Of Manuel M T Chakravarty >> Sent: 06 June 2018 12:50 >> To: Simon Marlow >> Cc: ghc-devops-group at haskell.org >> Subject: Re: [GHC DevOps Group] State of CI >> >> >> Am 06.06.2018 um 19:11 schrieb Simon Marlow : >> >> >> * None of this work is GitHub specific. Nor all that CircleCI or >> Appveyor specific for that matter (work is currently focused on >> improving the test suite). >> * Our GitHub lock-in factor is currently low to pretty much absent, >> and would remain low even if the review workflow becomes more >> systematically GitHub centric (it already is for some small >> contributions). >> * That's because tickets remain on Trac, and the code along with the >> entirety of its history remains in a standard Git repository, GitHub >> or not. Also because GitHub is not a CI provider, those providers we >> do use integrate with other code hosting solutions (e.g. Appveyor with >> GitLab), and the surface area of CI provider-specific code is small. >> >> >> We should keep in mind, though, is that past code reviews is valuable content that we can't discard, nor can we easily migrate it to a different code review platform. At this point we have nearly 5K diffs on Phabricator, many of which have non-trivial code-review trails, and these are cross-referenced from Trac, emails, and other places. Even if we moved to github, we would want to keep Phabricator running so that we have access to this content, and people will experience friction though havng to deal with another system. >> >> To me, the friction caused by the transition and the inability to do a clean move is more worrying than the missing code review functionality on github. >> >> >> Actually, to me this is a red flag. Core reviews shouldn’t be essential documentation. I wonder what has been going wrong so that we have got to this situation. >> >> If important points are uncovered or documented in code reviews, that information should be included in either the source code or associated documentation. (Having git history as an important record for the evolution of the code is fine as it is now standard to not only have one snapshot of the source, but a source control history of the source as the definitive record of a piece of software. And as we painfully discover, much in contrast to code reviews, the source control history is recorded in a widely understood standard format.) >> >> Cheers, >> Manuel >> >> >> >> _______________________________________________ >> Ghc-devops-group mailing list >> Ghc-devops-group at haskell.org >> https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devops-group >> >> > > _______________________________________________ > Ghc-devops-group mailing list > Ghc-devops-group at haskell.org > https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devops-group From m at tweag.io Thu Jun 7 13:41:02 2018 From: m at tweag.io (Boespflug, Mathieu) Date: Thu, 7 Jun 2018 15:41:02 +0200 Subject: [GHC DevOps Group] State of CI In-Reply-To: References: <87zi09gpr6.fsf@smart-cactus.org> <64C0CA63-B54F-4973-A8A2-044106B74500@tweag.io> Message-ID: I feel neutral about whether Phabricator state ought to be considered ephemeral. But let's say we were to keep it all, in read-only form, when a switch to a different review tool occurs (after all, that projects, open or not, change review tools is a frequent occurrence). SimonM, would that mitigate in your view "the friction caused by the transition and the inability to do a clean move", which you mentioned in a previous email? -- Mathieu Boespflug Founder at http://tweag.io. On 7 June 2018 at 08:39, Manuel M T Chakravarty wrote: > I very much feel like SimonPJ about this. > > Moreover, I think, there are two important technical reasons to prefer > this style. Firstly, I don’t want to be dependent on any one code review > tool. The lock-in that Phabricator has been able to create here is quite > tiresome (even if Phacility is probably quite happy about the situation). > > Secondly, I’d be willing to bet that the history in code reviews is useful > only to very few people, namely those who participated in the respective > code reviews. This is in direct opposition to creating an open project, > which tries to level the playing field as far as possible. Hence, IMHO it > runs counter to the aims of this group. > > And, SimonM, as you bring up the differences between open-source and > closed source. I agree that this would be less of an issue in a closed > source project. However, I always thought, we want to make contributing to > GHC as easy as possible. > > Cheers, > Manuel > > Am 06.06.2018 um 23:47 schrieb Simon Peyton Jones via Ghc-devops-group < > ghc-devops-group at haskell.org>: > > Interestingly, I think this discussion has really teased out a key > difference - perhaps the reason people prefer different tools here is > because they're starting from different perspectives about what the tools > are for? > > I think you are right here. Personally, I find that Phab focuses my > attention on the **code**, rather than the design or architecture. Even > where discussions about the latter happen, they are buried in a sea of > noise about low-level suggestions. So my personal preference is to keep > the higher level stuff on Trac (or even a wiki page) and regard the code > review discussion as ephemeral. > > Simon > > *From:* Simon Marlow > *Sent:* 06 June 2018 13:44 > *To:* Simon Peyton Jones > *Cc:* Manuel M T Chakravarty ; > ghc-devops-group at haskell.org > *Subject:* Re: [GHC DevOps Group] State of CI > > > I think this reflects a different philosophy about code review. If we say > that code-review is only for small-scale suggestions about the actual code > changes, then yes I'd agree it's not all that useful to keep that history > around. But we can (and sometimes do) have high-level architectural > discussions as part of code review too, and indeed I think it's arguably > the right thing to have these discussion around concrete code proposals. We > keep all the discussion of the code in one place, and the discussion is > attached to the evolving code patches. In this world, the code discussions > really are valuable history. > > > > Interestingly, I think this discussion has really teased out a key > difference - perhaps the reason people prefer different tools here is > because they're starting from different perspectives about what the tools > are for? I don't think there's one true way here. Personally I've been > exposed to multiple different working styles, especially having one foot in > industry and one in the open-source world, and I've seen different > philosophies that work. We could as a project decide that we don't want to > go the way of having high-level discussion alongside code review, in which > case that's OK (I would slightly prefer to move in the other direction, but > that's just my preference). > > > > Cheers > > Simon > > > > > > On 6 June 2018 at 12:58, Simon Peyton Jones wrote: > > We should keep in mind, though, is that past code reviews is valuable > content that we can't discard, > > Like Manuel I’m not at all sure about this. > > I **do** regard the Trac conversation as a long-term asset, and often > refer to tickets from Notes, as a way to say “here’s a more extensive > discussion of what’s in the Note”. > > But the Phab discussion of “please refactor this or that” seems far less > valuable. And actually I don’t think it is substantially cross-referenced > from elsewhere. Where there is substantive conversation about the > approach, I’d rather see that on Trac. > > So for me, long term access to the code-review trail is not very important > > Simon > > *From:* Ghc-devops-group *On > Behalf Of *Manuel M T Chakravarty > *Sent:* 06 June 2018 12:50 > *To:* Simon Marlow > *Cc:* ghc-devops-group at haskell.org > *Subject:* Re: [GHC DevOps Group] State of CI > > > Am 06.06.2018 um 19:11 schrieb Simon Marlow : > > > * None of this work is GitHub specific. Nor all that CircleCI or > Appveyor specific for that matter (work is currently focused on > improving the test suite). > * Our GitHub lock-in factor is currently low to pretty much absent, > and would remain low even if the review workflow becomes more > systematically GitHub centric (it already is for some small > contributions). > * That's because tickets remain on Trac, and the code along with the > entirety of its history remains in a standard Git repository, GitHub > or not. Also because GitHub is not a CI provider, those providers we > do use integrate with other code hosting solutions (e.g. Appveyor with > GitLab), and the surface area of CI provider-specific code is small. > > > We should keep in mind, though, is that past code reviews is valuable > content that we can't discard, nor can we easily migrate it to a different > code review platform. At this point we have nearly 5K diffs on Phabricator, > many of which have non-trivial code-review trails, and these are > cross-referenced from Trac, emails, and other places. Even if we moved to > github, we would want to keep Phabricator running so that we have access to > this content, and people will experience friction though havng to deal with > another system. > > To me, the friction caused by the transition and the inability to do a > clean move is more worrying than the missing code review functionality on > github. > > > Actually, to me this is a red flag. Core reviews shouldn’t be essential > documentation. I wonder what has been going wrong so that we have got to > this situation. > > If important points are uncovered or documented in code reviews, that > information should be included in either the source code or associated > documentation. (Having git history as an important record for the evolution > of the code is fine as it is now standard to not only have one snapshot of > the source, but a source control history of the source as the definitive > record of a piece of software. And as we painfully discover, much in > contrast to code reviews, the source control history is recorded in a > widely understood standard format.) > > Cheers, > Manuel > > > > _______________________________________________ > Ghc-devops-group mailing list > Ghc-devops-group at haskell.org > https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devops-group > > > > _______________________________________________ > Ghc-devops-group mailing list > Ghc-devops-group at haskell.org > https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devops-group > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hvriedel at gmail.com Thu Jun 7 14:14:41 2018 From: hvriedel at gmail.com (Herbert Valerio Riedel) Date: Thu, 7 Jun 2018 16:14:41 +0200 Subject: [GHC DevOps Group] State of CI In-Reply-To: References: <87zi09gpr6.fsf@smart-cactus.org> <64C0CA63-B54F-4973-A8A2-044106B74500@tweag.io> Message-ID: Mathieu, ...while from my personal experience, changing code review tools is definitely *not* a frequent occurrence for large projects (open or not)... why are we even debating a potential change of review tools, if we haven't even found anything that's remotely comparable to Phabricator (i.e. as good or even better) and which we'd be able to self-host (to be able to retain permalinks to the information in case we'd switch yet again for whatever reasons)? What's the technical problem we're trying to solve here? -- hvr On Thu, Jun 7, 2018 at 3:41 PM Boespflug, Mathieu wrote: > > I feel neutral about whether Phabricator state ought to be considered ephemeral. But let's say we were to keep it all, in read-only form, when a switch to a different review tool occurs (after all, that projects, open or not, change review tools is a frequent occurrence). SimonM, would that mitigate in your view "the friction caused by the transition and the inability to do a clean move", which you mentioned in a previous email? > > -- > Mathieu Boespflug > Founder at http://tweag.io. > > On 7 June 2018 at 08:39, Manuel M T Chakravarty wrote: >> >> I very much feel like SimonPJ about this. >> >> Moreover, I think, there are two important technical reasons to prefer this style. Firstly, I don’t want to be dependent on any one code review tool. The lock-in that Phabricator has been able to create here is quite tiresome (even if Phacility is probably quite happy about the situation). >> >> Secondly, I’d be willing to bet that the history in code reviews is useful only to very few people, namely those who participated in the respective code reviews. This is in direct opposition to creating an open project, which tries to level the playing field as far as possible. Hence, IMHO it runs counter to the aims of this group. >> >> And, SimonM, as you bring up the differences between open-source and closed source. I agree that this would be less of an issue in a closed source project. However, I always thought, we want to make contributing to GHC as easy as possible. >> >> Cheers, >> Manuel >> >> Am 06.06.2018 um 23:47 schrieb Simon Peyton Jones via Ghc-devops-group : >> >> Interestingly, I think this discussion has really teased out a key difference - perhaps the reason people prefer different tools here is because they're starting from different perspectives about what the tools are for? >> >> I think you are right here. Personally, I find that Phab focuses my attention on the *code*, rather than the design or architecture. Even where discussions about the latter happen, they are buried in a sea of noise about low-level suggestions. So my personal preference is to keep the higher level stuff on Trac (or even a wiki page) and regard the code review discussion as ephemeral. >> >> Simon >> >> From: Simon Marlow >> Sent: 06 June 2018 13:44 >> To: Simon Peyton Jones >> Cc: Manuel M T Chakravarty ; ghc-devops-group at haskell.org >> Subject: Re: [GHC DevOps Group] State of CI >> >> >> I think this reflects a different philosophy about code review. If we say that code-review is only for small-scale suggestions about the actual code changes, then yes I'd agree it's not all that useful to keep that history around. But we can (and sometimes do) have high-level architectural discussions as part of code review too, and indeed I think it's arguably the right thing to have these discussion around concrete code proposals. We keep all the discussion of the code in one place, and the discussion is attached to the evolving code patches. In this world, the code discussions really are valuable history. >> >> >> >> Interestingly, I think this discussion has really teased out a key difference - perhaps the reason people prefer different tools here is because they're starting from different perspectives about what the tools are for? I don't think there's one true way here. Personally I've been exposed to multiple different working styles, especially having one foot in industry and one in the open-source world, and I've seen different philosophies that work. We could as a project decide that we don't want to go the way of having high-level discussion alongside code review, in which case that's OK (I would slightly prefer to move in the other direction, but that's just my preference). >> >> >> >> Cheers >> >> Simon >> >> >> >> >> >> On 6 June 2018 at 12:58, Simon Peyton Jones wrote: >> >> We should keep in mind, though, is that past code reviews is valuable content that we can't discard, >> >> Like Manuel I’m not at all sure about this. >> >> I *do* regard the Trac conversation as a long-term asset, and often refer to tickets from Notes, as a way to say “here’s a more extensive discussion of what’s in the Note”. >> >> But the Phab discussion of “please refactor this or that” seems far less valuable. And actually I don’t think it is substantially cross-referenced from elsewhere. Where there is substantive conversation about the approach, I’d rather see that on Trac. >> >> So for me, long term access to the code-review trail is not very important >> >> Simon >> >> From: Ghc-devops-group On Behalf Of Manuel M T Chakravarty >> Sent: 06 June 2018 12:50 >> To: Simon Marlow >> Cc: ghc-devops-group at haskell.org >> Subject: Re: [GHC DevOps Group] State of CI >> >> >> Am 06.06.2018 um 19:11 schrieb Simon Marlow : >> >> >> * None of this work is GitHub specific. Nor all that CircleCI or >> Appveyor specific for that matter (work is currently focused on >> improving the test suite). >> * Our GitHub lock-in factor is currently low to pretty much absent, >> and would remain low even if the review workflow becomes more >> systematically GitHub centric (it already is for some small >> contributions). >> * That's because tickets remain on Trac, and the code along with the >> entirety of its history remains in a standard Git repository, GitHub >> or not. Also because GitHub is not a CI provider, those providers we >> do use integrate with other code hosting solutions (e.g. Appveyor with >> GitLab), and the surface area of CI provider-specific code is small. >> >> >> We should keep in mind, though, is that past code reviews is valuable content that we can't discard, nor can we easily migrate it to a different code review platform. At this point we have nearly 5K diffs on Phabricator, many of which have non-trivial code-review trails, and these are cross-referenced from Trac, emails, and other places. Even if we moved to github, we would want to keep Phabricator running so that we have access to this content, and people will experience friction though havng to deal with another system. >> >> To me, the friction caused by the transition and the inability to do a clean move is more worrying than the missing code review functionality on github. >> >> >> Actually, to me this is a red flag. Core reviews shouldn’t be essential documentation. I wonder what has been going wrong so that we have got to this situation. >> >> If important points are uncovered or documented in code reviews, that information should be included in either the source code or associated documentation. (Having git history as an important record for the evolution of the code is fine as it is now standard to not only have one snapshot of the source, but a source control history of the source as the definitive record of a piece of software. And as we painfully discover, much in contrast to code reviews, the source control history is recorded in a widely understood standard format.) >> >> Cheers, >> Manuel >> >> >> >> _______________________________________________ >> Ghc-devops-group mailing list >> Ghc-devops-group at haskell.org >> https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devops-group >> >> >> >> _______________________________________________ >> Ghc-devops-group mailing list >> Ghc-devops-group at haskell.org >> https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devops-group >> > > _______________________________________________ > Ghc-devops-group mailing list > Ghc-devops-group at haskell.org > https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devops-group From manuel.chakravarty at tweag.io Thu Jun 7 16:37:56 2018 From: manuel.chakravarty at tweag.io (Manuel M T Chakravarty) Date: Fri, 8 Jun 2018 00:37:56 +0800 Subject: [GHC DevOps Group] State of CI In-Reply-To: References: <87zi09gpr6.fsf@smart-cactus.org> <64C0CA63-B54F-4973-A8A2-044106B74500@tweag.io> Message-ID: <5F30C32E-4A9B-4182-A453-47167DDC0005@tweag.io> Sorry if I didn’t clearly state my second point. I do not doubt that these reviews are useful to you (or other people involved in the actual code reviews). I am saying that they are much less accessible than a discussion on Trac or, better even, documentation on the wiki for everybody else. They are *bad* documentation as (1) they lock us into a review tool with no standardised data format and (2) they serve a few rather than many. Cheers, Manuel > Am 07.06.2018 um 16:22 schrieb Simon Marlow : > > The first point I think is reasonable, being independent of any one code review tool would be an advantage. However, I'm prepared to sacrifice that, because I think the conclusion it leads to is suboptimal - that code review discussion is ephemeral, and we cannot rely on having access to it after the code is committed. > > To make this concrete, I picked out some examples of diffs I've been involved in recently. I'm sure there are plenty more examples like this: > > https://phabricator.haskell.org/D42 (click on "Show Older Comments" to see all the discussion) > https://phabricator.haskell.org/D4327 > https://phabricator.haskell.org/D4324 > https://phabricator.haskell.org/D4679 > https://phabricator.haskell.org/D4644 > https://phabricator.haskell.org/D4722 > https://phabricator.haskell.org/D4302 > > The discussion here is substantial and I would be really sad to lose it, and I think its relevance goes beyond just the people involved in the review. Note the cross-referencing between diffs, the links to code, the comments on particular code fragments, and links to pastes hosted by Phabricator. It's definitely a lot more than just small-scale suggestions for improvements or refactoring. > > We also have cross-references from Trac tickets to Phabricator diffs, and every diff committed contains a link to the code review (I use these links a lot). There are links to Phabricator from mailing lists and wiki pages. > > So we could stop doing this, and discuss only on Trac tickets or mailing lists. (wiki is not good for discussion). After all, that's what we used to do before code review, and we managed. But I think to do that would be a loss - GHC development has felt a lot more engaging since we added code review, and restricting it to ephemeral discussion would curtail its potential dramatically. I'd have to think twice before writing a comment on a diff - if I want to discuss something about the overall approach, I'd have to leave a link to Trac (and create a ticket if there wasn't one), which is a huge faff. > > Ok, so I've made my case. This is not me stamping my foot and saying "we must do it this way" by any means! If there's consensus to change things then I'll happily adjust my workflows. > > Cheers > Simon > > > > > On 7 June 2018 at 07:39, Manuel M T Chakravarty > wrote: > I very much feel like SimonPJ about this. > > Moreover, I think, there are two important technical reasons to prefer this style. Firstly, I don’t want to be dependent on any one code review tool. The lock-in that Phabricator has been able to create here is quite tiresome (even if Phacility is probably quite happy about the situation). > > Secondly, I’d be willing to bet that the history in code reviews is useful only to very few people, namely those who participated in the respective code reviews. This is in direct opposition to creating an open project, which tries to level the playing field as far as possible. Hence, IMHO it runs counter to the aims of this group. > > And, SimonM, as you bring up the differences between open-source and closed source. I agree that this would be less of an issue in a closed source project. However, I always thought, we want to make contributing to GHC as easy as possible. > > Cheers, > Manuel > >> Am 06.06.2018 um 23:47 schrieb Simon Peyton Jones via Ghc-devops-group >: >> >> Interestingly, I think this discussion has really teased out a key difference - perhaps the reason people prefer different tools here is because they're starting from different perspectives about what the tools are for? >> >> I think you are right here. Personally, I find that Phab focuses my attention on the *code*, rather than the design or architecture. Even where discussions about the latter happen, they are buried in a sea of noise about low-level suggestions. So my personal preference is to keep the higher level stuff on Trac (or even a wiki page) and regard the code review discussion as ephemeral. >> >> Simon >> >> From: Simon Marlow > >> Sent: 06 June 2018 13:44 >> To: Simon Peyton Jones > >> Cc: Manuel M T Chakravarty >; ghc-devops-group at haskell.org >> Subject: Re: [GHC DevOps Group] State of CI >> >> I think this reflects a different philosophy about code review. If we say that code-review is only for small-scale suggestions about the actual code changes, then yes I'd agree it's not all that useful to keep that history around. But we can (and sometimes do) have high-level architectural discussions as part of code review too, and indeed I think it's arguably the right thing to have these discussion around concrete code proposals. We keep all the discussion of the code in one place, and the discussion is attached to the evolving code patches. In this world, the code discussions really are valuable history. >> >> >> >> Interestingly, I think this discussion has really teased out a key difference - perhaps the reason people prefer different tools here is because they're starting from different perspectives about what the tools are for? I don't think there's one true way here. Personally I've been exposed to multiple different working styles, especially having one foot in industry and one in the open-source world, and I've seen different philosophies that work. We could as a project decide that we don't want to go the way of having high-level discussion alongside code review, in which case that's OK (I would slightly prefer to move in the other direction, but that's just my preference). >> >> >> >> Cheers >> >> Simon >> >> >> >> >> >> On 6 June 2018 at 12:58, Simon Peyton Jones > wrote: >> >> We should keep in mind, though, is that past code reviews is valuable content that we can't discard, >> >> Like Manuel I’m not at all sure about this. >> >> I *do* regard the Trac conversation as a long-term asset, and often refer to tickets from Notes, as a way to say “here’s a more extensive discussion of what’s in the Note”. >> >> But the Phab discussion of “please refactor this or that” seems far less valuable. And actually I don’t think it is substantially cross-referenced from elsewhere. Where there is substantive conversation about the approach, I’d rather see that on Trac. >> >> So for me, long term access to the code-review trail is not very important >> >> Simon >> >> From: Ghc-devops-group > On Behalf Of Manuel M T Chakravarty >> Sent: 06 June 2018 12:50 >> To: Simon Marlow > >> Cc: ghc-devops-group at haskell.org >> Subject: Re: [GHC DevOps Group] State of CI >> >> Am 06.06.2018 um 19:11 schrieb Simon Marlow >: >> >> * None of this work is GitHub specific. Nor all that CircleCI or >> Appveyor specific for that matter (work is currently focused on >> improving the test suite). >> * Our GitHub lock-in factor is currently low to pretty much absent, >> and would remain low even if the review workflow becomes more >> systematically GitHub centric (it already is for some small >> contributions). >> * That's because tickets remain on Trac, and the code along with the >> entirety of its history remains in a standard Git repository, GitHub >> or not. Also because GitHub is not a CI provider, those providers we >> do use integrate with other code hosting solutions (e.g. Appveyor with >> GitLab), and the surface area of CI provider-specific code is small. >> >> We should keep in mind, though, is that past code reviews is valuable content that we can't discard, nor can we easily migrate it to a different code review platform. At this point we have nearly 5K diffs on Phabricator, many of which have non-trivial code-review trails, and these are cross-referenced from Trac, emails, and other places. Even if we moved to github, we would want to keep Phabricator running so that we have access to this content, and people will experience friction though havng to deal with another system. >> >> To me, the friction caused by the transition and the inability to do a clean move is more worrying than the missing code review functionality on github. >> >> Actually, to me this is a red flag. Core reviews shouldn’t be essential documentation. I wonder what has been going wrong so that we have got to this situation. >> >> If important points are uncovered or documented in code reviews, that information should be included in either the source code or associated documentation. (Having git history as an important record for the evolution of the code is fine as it is now standard to not only have one snapshot of the source, but a source control history of the source as the definitive record of a piece of software. And as we painfully discover, much in contrast to code reviews, the source control history is recorded in a widely understood standard format.) >> >> Cheers, >> Manuel >> >> >> _______________________________________________ >> Ghc-devops-group mailing list >> Ghc-devops-group at haskell.org >> https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devops-group > -------------- next part -------------- An HTML attachment was scrubbed... URL: From marlowsd at gmail.com Fri Jun 8 10:49:25 2018 From: marlowsd at gmail.com (Simon Marlow) Date: Fri, 8 Jun 2018 11:49:25 +0100 Subject: [GHC DevOps Group] State of CI In-Reply-To: <5F30C32E-4A9B-4182-A453-47167DDC0005@tweag.io> References: <87zi09gpr6.fsf@smart-cactus.org> <64C0CA63-B54F-4973-A8A2-044106B74500@tweag.io> <5F30C32E-4A9B-4182-A453-47167DDC0005@tweag.io> Message-ID: With all due respect Manuel I'm afraid I continue to disagree :) I don't think anyone is claiming that code review is documentation, or should replace documentation. It's *discussion*, in the same sense as mailing list discussion. We keep mailing list archives because they provide useful context in the future and avoid us having to rediscover rationale, and we shouldn't throw away code reviews for the same reason. Moreover, the way that code review discussions are attached to actual code which lives in the repository makes them arguably even more useful and accessible. I think the examples below demonstrate that there's valuable content here, but not just for the people involved in that particular code review. It's a shared codebase after all, the next person to work on this code might come along ask "why did we end up with this design?". If the available docs/Notes don't tell the whole story, I can point to the code review discussion and we don't have to recreate the whole thing again. You could argue that the docs/Notes *should* tell the whole story - this point came up before - in practice they don't and the discussion is useful context. Furthermore, code review discussion is really conveniently accessible. Want to see the discussion that led up to a particular commit? The link is right there in the commit log! Finally, the data in Phabricator is no more locked-in than what we have on Trac: we have the DB and the open-source code that manipulates it. I don't think code-review data on github is accessible right now (at least I couldn't find a way to download it). Maybe it will be accessible in the future, but I think your argument is that we don't care because code reviews are ephemeral, which I don't agree with. Cheers Simon On 7 June 2018 at 17:37, Manuel M T Chakravarty wrote: > Sorry if I didn’t clearly state my second point. I do not doubt that these > reviews are useful to you (or other people involved in the actual code > reviews). I am saying that they are much less accessible than a discussion > on Trac or, better even, documentation on the wiki for everybody else. > > They are *bad* documentation as (1) they lock us into a review tool with > no standardised data format and (2) they serve a few rather than many. > > Cheers, > Manuel > > > Am 07.06.2018 um 16:22 schrieb Simon Marlow : > > The first point I think is reasonable, being independent of any one code > review tool would be an advantage. However, I'm prepared to sacrifice that, > because I think the conclusion it leads to is suboptimal - that code review > discussion is ephemeral, and we cannot rely on having access to it after > the code is committed. > > To make this concrete, I picked out some examples of diffs I've been > involved in recently. I'm sure there are plenty more examples like this: > > https://phabricator.haskell.org/D42 (click on "Show Older Comments" to > see all the discussion) > https://phabricator.haskell.org/D4327 > https://phabricator.haskell.org/D4324 > https://phabricator.haskell.org/D4679 > https://phabricator.haskell.org/D4644 > https://phabricator.haskell.org/D4722 > https://phabricator.haskell.org/D4302 > > The discussion here is substantial and I would be really sad to lose it, > and I think its relevance goes beyond just the people involved in the > review. Note the cross-referencing between diffs, the links to code, the > comments on particular code fragments, and links to pastes hosted by > Phabricator. It's definitely a lot more than just small-scale suggestions > for improvements or refactoring. > > We also have cross-references from Trac tickets to Phabricator diffs, and > every diff committed contains a link to the code review (I use these links > a lot). There are links to Phabricator from mailing lists and wiki pages. > > So we could stop doing this, and discuss only on Trac tickets or mailing > lists. (wiki is not good for discussion). After all, that's what we used to > do before code review, and we managed. But I think to do that would be a > loss - GHC development has felt a lot more engaging since we added code > review, and restricting it to ephemeral discussion would curtail its > potential dramatically. I'd have to think twice before writing a comment on > a diff - if I want to discuss something about the overall approach, I'd > have to leave a link to Trac (and create a ticket if there wasn't one), > which is a huge faff. > > Ok, so I've made my case. This is not me stamping my foot and saying "we > must do it this way" by any means! If there's consensus to change things > then I'll happily adjust my workflows. > > Cheers > Simon > > > > > On 7 June 2018 at 07:39, Manuel M T Chakravarty < > manuel.chakravarty at tweag.io> wrote: > >> I very much feel like SimonPJ about this. >> >> Moreover, I think, there are two important technical reasons to prefer >> this style. Firstly, I don’t want to be dependent on any one code review >> tool. The lock-in that Phabricator has been able to create here is quite >> tiresome (even if Phacility is probably quite happy about the situation). >> >> Secondly, I’d be willing to bet that the history in code reviews is >> useful only to very few people, namely those who participated in the >> respective code reviews. This is in direct opposition to creating an open >> project, which tries to level the playing field as far as possible. Hence, >> IMHO it runs counter to the aims of this group. >> >> And, SimonM, as you bring up the differences between open-source and >> closed source. I agree that this would be less of an issue in a closed >> source project. However, I always thought, we want to make contributing to >> GHC as easy as possible. >> >> Cheers, >> Manuel >> >> Am 06.06.2018 um 23:47 schrieb Simon Peyton Jones via Ghc-devops-group < >> ghc-devops-group at haskell.org>: >> >> Interestingly, I think this discussion has really teased out a key >> difference - perhaps the reason people prefer different tools here is >> because they're starting from different perspectives about what the tools >> are for? >> >> I think you are right here. Personally, I find that Phab focuses my >> attention on the **code**, rather than the design or architecture. Even >> where discussions about the latter happen, they are buried in a sea of >> noise about low-level suggestions. So my personal preference is to keep >> the higher level stuff on Trac (or even a wiki page) and regard the code >> review discussion as ephemeral. >> >> Simon >> >> *From:* Simon Marlow >> *Sent:* 06 June 2018 13:44 >> *To:* Simon Peyton Jones >> *Cc:* Manuel M T Chakravarty ; >> ghc-devops-group at haskell.org >> *Subject:* Re: [GHC DevOps Group] State of CI >> >> >> I think this reflects a different philosophy about code review. If we say >> that code-review is only for small-scale suggestions about the actual code >> changes, then yes I'd agree it's not all that useful to keep that history >> around. But we can (and sometimes do) have high-level architectural >> discussions as part of code review too, and indeed I think it's arguably >> the right thing to have these discussion around concrete code proposals. We >> keep all the discussion of the code in one place, and the discussion is >> attached to the evolving code patches. In this world, the code discussions >> really are valuable history. >> >> >> >> Interestingly, I think this discussion has really teased out a key >> difference - perhaps the reason people prefer different tools here is >> because they're starting from different perspectives about what the tools >> are for? I don't think there's one true way here. Personally I've been >> exposed to multiple different working styles, especially having one foot in >> industry and one in the open-source world, and I've seen different >> philosophies that work. We could as a project decide that we don't want to >> go the way of having high-level discussion alongside code review, in which >> case that's OK (I would slightly prefer to move in the other direction, but >> that's just my preference). >> >> >> >> Cheers >> >> Simon >> >> >> >> >> >> On 6 June 2018 at 12:58, Simon Peyton Jones >> wrote: >> >> We should keep in mind, though, is that past code reviews is valuable >> content that we can't discard, >> >> Like Manuel I’m not at all sure about this. >> >> I **do** regard the Trac conversation as a long-term asset, and often >> refer to tickets from Notes, as a way to say “here’s a more extensive >> discussion of what’s in the Note”. >> >> But the Phab discussion of “please refactor this or that” seems far less >> valuable. And actually I don’t think it is substantially cross-referenced >> from elsewhere. Where there is substantive conversation about the >> approach, I’d rather see that on Trac. >> >> So for me, long term access to the code-review trail is not very important >> >> Simon >> >> *From:* Ghc-devops-group *On >> Behalf Of *Manuel M T Chakravarty >> *Sent:* 06 June 2018 12:50 >> *To:* Simon Marlow >> *Cc:* ghc-devops-group at haskell.org >> *Subject:* Re: [GHC DevOps Group] State of CI >> >> >> Am 06.06.2018 um 19:11 schrieb Simon Marlow : >> >> >> * None of this work is GitHub specific. Nor all that CircleCI or >> Appveyor specific for that matter (work is currently focused on >> improving the test suite). >> * Our GitHub lock-in factor is currently low to pretty much absent, >> and would remain low even if the review workflow becomes more >> systematically GitHub centric (it already is for some small >> contributions). >> * That's because tickets remain on Trac, and the code along with the >> entirety of its history remains in a standard Git repository, GitHub >> or not. Also because GitHub is not a CI provider, those providers we >> do use integrate with other code hosting solutions (e.g. Appveyor with >> GitLab), and the surface area of CI provider-specific code is small. >> >> >> We should keep in mind, though, is that past code reviews is valuable >> content that we can't discard, nor can we easily migrate it to a different >> code review platform. At this point we have nearly 5K diffs on Phabricator, >> many of which have non-trivial code-review trails, and these are >> cross-referenced from Trac, emails, and other places. Even if we moved to >> github, we would want to keep Phabricator running so that we have access to >> this content, and people will experience friction though havng to deal with >> another system. >> >> To me, the friction caused by the transition and the inability to do a >> clean move is more worrying than the missing code review functionality on >> github. >> >> >> Actually, to me this is a red flag. Core reviews shouldn’t be essential >> documentation. I wonder what has been going wrong so that we have got to >> this situation. >> >> If important points are uncovered or documented in code reviews, that >> information should be included in either the source code or associated >> documentation. (Having git history as an important record for the evolution >> of the code is fine as it is now standard to not only have one snapshot of >> the source, but a source control history of the source as the definitive >> record of a piece of software. And as we painfully discover, much in >> contrast to code reviews, the source control history is recorded in a >> widely understood standard format.) >> >> Cheers, >> Manuel >> >> >> >> _______________________________________________ >> Ghc-devops-group mailing list >> Ghc-devops-group at haskell.org >> https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devops-group >> >> >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben at well-typed.com Fri Jun 15 17:13:54 2018 From: ben at well-typed.com (Ben Gamari) Date: Fri, 15 Jun 2018 13:13:54 -0400 Subject: [GHC DevOps Group] CircleCI status and budget concerns Message-ID: <87h8m4rn01.fsf@smart-cactus.org> Hi everyone, Over the last few weeks while finalizing the 8.6 branch I've been working on cleaning up some loose ends on the CI front. This includes, * Fixing the Fedora and Hadrian builds * Whittling away at the i386 Linux failures * Incorporating nightly slow validation (which is now green, thanks to Alp's efforts) * Incorporating Alp's fixes to Hadrian to get this build green again During this process, however, I have noticed that a concerning banner has appeared at the top of the CircleCI interface. It currently reads: Your current usage represents 84% of your ghc Linux plan's limit. Please upgrade in order to ensure no disruption in building. With the percentage gradually ticking upwards as builds finish. I believe 84% refers to our plan's container time allotment of 1500 hours (confusingly mislabeled as "1500 minutes" on the website). So far in the current May 21 - Jun 21 billing period we have used 1250 hours of build time. This is concerning as it presumably means that when we eventually overrun our budget we will be left without any CI at all for the remainder of the month. Given that we are currently building only a subset of commits on only the `master` I'm rather concerned that we are within a stone's throw of this situation (perhaps even going over this month). This seems untenable, especially when we consider that the number of builds will soon be growing to include stable branch backports, patches under review, and release builds. To manage this it seems that we will likely need to draw from a few of the tools at our disposal: a. Reduce the number of build configurations that we build every commit in (e.g. Fedora and LLVM could likely be relegated to a nightly build) b. Reduce the amount of parallelism that we allow CircleCI to employ: having access to CI consistently throughout the month is far preferable to faster build turnarounds for some of the month followed by periods of no CI for the rest. c. Increase the build time limit; this is $50/month/container In the short-run we can implement (a) immediately. This of course will reduce our coverage, but it looks like our coverage goals may not be practical given the resources we have. I've looked into (b) and haven't found any indication that CircleCI supports our use-case. If this is the case then we will have to simply pursue (c). I suppose haskell.org may be able to contribute here. However, I wonder what other resources could be deployed. Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From alex at cloudware.io Fri Jun 15 17:22:06 2018 From: alex at cloudware.io (alex) Date: Fri, 15 Jun 2018 19:22:06 +0200 Subject: [GHC DevOps Group] CircleCI status and budget concerns In-Reply-To: <87h8m4rn01.fsf@smart-cactus.org> References: <87h8m4rn01.fsf@smart-cactus.org> Message-ID: > 84% refers to our plan's container time allotment of 1500 hours Even though Circle CI is using Google Cloud instances, the CI still imposes a time limit? But why? On 15 June 2018 at 19:13, Ben Gamari wrote: > Hi everyone, > > Over the last few weeks while finalizing the 8.6 branch I've been > working on cleaning up some loose ends on the CI front. This includes, > > * Fixing the Fedora and Hadrian builds > * Whittling away at the i386 Linux failures > * Incorporating nightly slow validation (which is now green, thanks to > Alp's efforts) > * Incorporating Alp's fixes to Hadrian to get this build green again > > During this process, however, I have noticed that a concerning banner > has appeared at the top of the CircleCI interface. It currently reads: > > Your current usage represents 84% of your ghc Linux plan's limit. > Please upgrade in order to ensure no disruption in building. > > With the percentage gradually ticking upwards as builds finish. I > believe 84% refers to our plan's container time allotment of 1500 > hours (confusingly mislabeled as "1500 minutes" on the website). So far > in the current May 21 - Jun 21 billing period we have used 1250 hours of > build time. This is concerning as it presumably means that when we > eventually overrun our budget we will be left without any CI at all for > the remainder of the month. > > Given that we are currently building only a subset of commits on only > the `master` I'm rather concerned that we are within a stone's throw of > this situation (perhaps even going over this month). This seems > untenable, especially when we consider that the number of builds will > soon be growing to include stable branch backports, patches under > review, and release builds. > > To manage this it seems that we will likely need to draw from a few of > the tools at our disposal: > > a. Reduce the number of build configurations that we build every commit > in (e.g. Fedora and LLVM could likely be relegated to a nightly build) > > b. Reduce the amount of parallelism that we allow CircleCI to employ: > having access to CI consistently throughout the month is far > preferable to faster build turnarounds for some of the month > followed by periods of no CI for the rest. > > c. Increase the build time limit; this is $50/month/container > > In the short-run we can implement (a) immediately. This of course will > reduce our coverage, but it looks like our coverage goals may not be > practical given the resources we have. I've looked into (b) and haven't > found any indication that CircleCI supports our use-case. If this is > the case then we will have to simply pursue (c). > > I suppose haskell.org may be able to contribute here. However, I wonder > what other resources could be deployed. > > Cheers, > > - Ben > > _______________________________________________ > Ghc-devops-group mailing list > Ghc-devops-group at haskell.org > https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devops-group > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From alan.zimm at gmail.com Fri Jun 15 17:23:57 2018 From: alan.zimm at gmail.com (Alan & Kim Zimmerman) Date: Fri, 15 Jun 2018 19:23:57 +0200 Subject: [GHC DevOps Group] CircleCI status and budget concerns In-Reply-To: References: <87h8m4rn01.fsf@smart-cactus.org> Message-ID: I know the gitlab CI process allows you to use external resources, so we could use our own instances. On 15 June 2018 at 19:22, alex wrote: > > 84% refers to our plan's container time allotment of 1500 hours > > Even though Circle CI is using Google Cloud instances, the CI still > imposes a time limit? But why? > > On 15 June 2018 at 19:13, Ben Gamari wrote: > >> Hi everyone, >> >> Over the last few weeks while finalizing the 8.6 branch I've been >> working on cleaning up some loose ends on the CI front. This includes, >> >> * Fixing the Fedora and Hadrian builds >> * Whittling away at the i386 Linux failures >> * Incorporating nightly slow validation (which is now green, thanks to >> Alp's efforts) >> * Incorporating Alp's fixes to Hadrian to get this build green again >> >> During this process, however, I have noticed that a concerning banner >> has appeared at the top of the CircleCI interface. It currently reads: >> >> Your current usage represents 84% of your ghc Linux plan's limit. >> Please upgrade in order to ensure no disruption in building. >> >> With the percentage gradually ticking upwards as builds finish. I >> believe 84% refers to our plan's container time allotment of 1500 >> hours (confusingly mislabeled as "1500 minutes" on the website). So far >> in the current May 21 - Jun 21 billing period we have used 1250 hours of >> build time. This is concerning as it presumably means that when we >> eventually overrun our budget we will be left without any CI at all for >> the remainder of the month. >> >> Given that we are currently building only a subset of commits on only >> the `master` I'm rather concerned that we are within a stone's throw of >> this situation (perhaps even going over this month). This seems >> untenable, especially when we consider that the number of builds will >> soon be growing to include stable branch backports, patches under >> review, and release builds. >> >> To manage this it seems that we will likely need to draw from a few of >> the tools at our disposal: >> >> a. Reduce the number of build configurations that we build every commit >> in (e.g. Fedora and LLVM could likely be relegated to a nightly build) >> >> b. Reduce the amount of parallelism that we allow CircleCI to employ: >> having access to CI consistently throughout the month is far >> preferable to faster build turnarounds for some of the month >> followed by periods of no CI for the rest. >> >> c. Increase the build time limit; this is $50/month/container >> >> In the short-run we can implement (a) immediately. This of course will >> reduce our coverage, but it looks like our coverage goals may not be >> practical given the resources we have. I've looked into (b) and haven't >> found any indication that CircleCI supports our use-case. If this is >> the case then we will have to simply pursue (c). >> >> I suppose haskell.org may be able to contribute here. However, I wonder >> what other resources could be deployed. >> >> Cheers, >> >> - Ben >> >> _______________________________________________ >> Ghc-devops-group mailing list >> Ghc-devops-group at haskell.org >> https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devops-group >> >> > > _______________________________________________ > Ghc-devops-group mailing list > Ghc-devops-group at haskell.org > https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devops-group > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gnezdo at google.com Fri Jun 15 17:28:16 2018 From: gnezdo at google.com (Greg Steuck (Sh-toy-k)) Date: Fri, 15 Jun 2018 10:28:16 -0700 Subject: [GHC DevOps Group] CircleCI status and budget concerns In-Reply-To: References: <87h8m4rn01.fsf@smart-cactus.org> Message-ID: We have some slack in the GHC VM donation GCP project if that can be used to run Circle CI. On Fri, Jun 15, 2018 at 10:24 AM Alan & Kim Zimmerman wrote: > I know the gitlab CI process allows you to use external resources, so we > could use our own instances. > > On 15 June 2018 at 19:22, alex wrote: > >> > 84% refers to our plan's container time allotment of 1500 hours >> >> Even though Circle CI is using Google Cloud instances, the CI still >> imposes a time limit? But why? >> >> On 15 June 2018 at 19:13, Ben Gamari wrote: >> >>> Hi everyone, >>> >>> Over the last few weeks while finalizing the 8.6 branch I've been >>> working on cleaning up some loose ends on the CI front. This includes, >>> >>> * Fixing the Fedora and Hadrian builds >>> * Whittling away at the i386 Linux failures >>> * Incorporating nightly slow validation (which is now green, thanks to >>> Alp's efforts) >>> * Incorporating Alp's fixes to Hadrian to get this build green again >>> >>> During this process, however, I have noticed that a concerning banner >>> has appeared at the top of the CircleCI interface. It currently reads: >>> >>> Your current usage represents 84% of your ghc Linux plan's limit. >>> Please upgrade in order to ensure no disruption in building. >>> >>> With the percentage gradually ticking upwards as builds finish. I >>> believe 84% refers to our plan's container time allotment of 1500 >>> hours (confusingly mislabeled as "1500 minutes" on the website). So far >>> in the current May 21 - Jun 21 billing period we have used 1250 hours of >>> build time. This is concerning as it presumably means that when we >>> eventually overrun our budget we will be left without any CI at all for >>> the remainder of the month. >>> >>> Given that we are currently building only a subset of commits on only >>> the `master` I'm rather concerned that we are within a stone's throw of >>> this situation (perhaps even going over this month). This seems >>> untenable, especially when we consider that the number of builds will >>> soon be growing to include stable branch backports, patches under >>> review, and release builds. >>> >>> To manage this it seems that we will likely need to draw from a few of >>> the tools at our disposal: >>> >>> a. Reduce the number of build configurations that we build every commit >>> in (e.g. Fedora and LLVM could likely be relegated to a nightly >>> build) >>> >>> b. Reduce the amount of parallelism that we allow CircleCI to employ: >>> having access to CI consistently throughout the month is far >>> preferable to faster build turnarounds for some of the month >>> followed by periods of no CI for the rest. >>> >>> c. Increase the build time limit; this is $50/month/container >>> >>> In the short-run we can implement (a) immediately. This of course will >>> reduce our coverage, but it looks like our coverage goals may not be >>> practical given the resources we have. I've looked into (b) and haven't >>> found any indication that CircleCI supports our use-case. If this is >>> the case then we will have to simply pursue (c). >>> >>> I suppose haskell.org may be able to contribute here. However, I wonder >>> what other resources could be deployed. >>> >>> Cheers, >>> >>> - Ben >>> >>> _______________________________________________ >>> Ghc-devops-group mailing list >>> Ghc-devops-group at haskell.org >>> https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devops-group >>> >>> >> >> _______________________________________________ >> Ghc-devops-group mailing list >> Ghc-devops-group at haskell.org >> https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devops-group >> >> > _______________________________________________ > Ghc-devops-group mailing list > Ghc-devops-group at haskell.org > https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devops-group > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4843 bytes Desc: S/MIME Cryptographic Signature URL: From manuel.chakravarty at tweag.io Mon Jun 18 04:04:20 2018 From: manuel.chakravarty at tweag.io (Manuel M T Chakravarty) Date: Mon, 18 Jun 2018 14:04:20 +1000 Subject: [GHC DevOps Group] CircleCI status and budget concerns In-Reply-To: <87h8m4rn01.fsf@smart-cactus.org> References: <87h8m4rn01.fsf@smart-cactus.org> Message-ID: Hi Ben, Thanks for bringing this up. I agree that (a) is useful short-term fix, but I think, the only reasonable long-term option is (c). After all, one reason to go with a hosted CI solution was to be able to solve this kind of problem with financial support by one of the stakeholders. How many containers have we got? (I think, one needs GitHub org admin access to be able to see that.) Cheers, Manuel > Am 16.06.2018 um 03:13 schrieb Ben Gamari : > > Signierter PGP-Teil > Hi everyone, > > Over the last few weeks while finalizing the 8.6 branch I've been > working on cleaning up some loose ends on the CI front. This includes, > > * Fixing the Fedora and Hadrian builds > * Whittling away at the i386 Linux failures > * Incorporating nightly slow validation (which is now green, thanks to > Alp's efforts) > * Incorporating Alp's fixes to Hadrian to get this build green again > > During this process, however, I have noticed that a concerning banner > has appeared at the top of the CircleCI interface. It currently reads: > > Your current usage represents 84% of your ghc Linux plan's limit. > Please upgrade in order to ensure no disruption in building. > > With the percentage gradually ticking upwards as builds finish. I > believe 84% refers to our plan's container time allotment of 1500 > hours (confusingly mislabeled as "1500 minutes" on the website). So far > in the current May 21 - Jun 21 billing period we have used 1250 hours of > build time. This is concerning as it presumably means that when we > eventually overrun our budget we will be left without any CI at all for > the remainder of the month. > > Given that we are currently building only a subset of commits on only > the `master` I'm rather concerned that we are within a stone's throw of > this situation (perhaps even going over this month). This seems > untenable, especially when we consider that the number of builds will > soon be growing to include stable branch backports, patches under > review, and release builds. > > To manage this it seems that we will likely need to draw from a few of > the tools at our disposal: > > a. Reduce the number of build configurations that we build every commit > in (e.g. Fedora and LLVM could likely be relegated to a nightly build) > > b. Reduce the amount of parallelism that we allow CircleCI to employ: > having access to CI consistently throughout the month is far > preferable to faster build turnarounds for some of the month > followed by periods of no CI for the rest. > > c. Increase the build time limit; this is $50/month/container > > In the short-run we can implement (a) immediately. This of course will > reduce our coverage, but it looks like our coverage goals may not be > practical given the resources we have. I've looked into (b) and haven't > found any indication that CircleCI supports our use-case. If this is > the case then we will have to simply pursue (c). > > I suppose haskell.org may be able to contribute here. However, I wonder > what other resources could be deployed. > > Cheers, > > - Ben > > From simonpj at microsoft.com Mon Jun 18 08:28:04 2018 From: simonpj at microsoft.com (Simon Peyton Jones) Date: Mon, 18 Jun 2018 08:28:04 +0000 Subject: [GHC DevOps Group] CircleCI status and budget concerns In-Reply-To: References: <87h8m4rn01.fsf@smart-cactus.org> Message-ID: | I agree that (a) is useful short-term fix, but I think, the only reasonable | long-term option is (c). After all, one reason to go with a hosted CI | solution was to be able to solve this kind of problem with financial support | by one of the stakeholders. I agree with Manuel. $50/month is *miniscule* when compared to the cost of person-hours required to work around the problem. If we can solve problems with very small amounts of money, that's a gift -- we should grab it! We have lots of problems that can't be so simply solved 😊. Simon | -----Original Message----- | From: Ghc-devops-group On Behalf Of | Manuel M T Chakravarty | Sent: 18 June 2018 05:04 | To: Ben Gamari | Cc: GHC DevOps Group | Subject: Re: [GHC DevOps Group] CircleCI status and budget concerns | | Hi Ben, | | Thanks for bringing this up. | | I agree that (a) is useful short-term fix, but I think, the only reasonable | long-term option is (c). After all, one reason to go with a hosted CI | solution was to be able to solve this kind of problem with financial support | by one of the stakeholders. | | How many containers have we got? (I think, one needs GitHub org admin access | to be able to see that.) | | Cheers, | Manuel | | > Am 16.06.2018 um 03:13 schrieb Ben Gamari : | > | > Signierter PGP-Teil | > Hi everyone, | > | > Over the last few weeks while finalizing the 8.6 branch I've been | > working on cleaning up some loose ends on the CI front. This includes, | > | > * Fixing the Fedora and Hadrian builds | > * Whittling away at the i386 Linux failures | > * Incorporating nightly slow validation (which is now green, thanks to | > Alp's efforts) | > * Incorporating Alp's fixes to Hadrian to get this build green again | > | > During this process, however, I have noticed that a concerning banner | > has appeared at the top of the CircleCI interface. It currently reads: | > | > Your current usage represents 84% of your ghc Linux plan's limit. | > Please upgrade in order to ensure no disruption in building. | > | > With the percentage gradually ticking upwards as builds finish. I | > believe 84% refers to our plan's container time allotment of 1500 | > hours (confusingly mislabeled as "1500 minutes" on the website). So | > far in the current May 21 - Jun 21 billing period we have used 1250 | > hours of build time. This is concerning as it presumably means that | > when we eventually overrun our budget we will be left without any CI | > at all for the remainder of the month. | > | > Given that we are currently building only a subset of commits on only | > the `master` I'm rather concerned that we are within a stone's throw | > of this situation (perhaps even going over this month). This seems | > untenable, especially when we consider that the number of builds will | > soon be growing to include stable branch backports, patches under | > review, and release builds. | > | > To manage this it seems that we will likely need to draw from a few of | > the tools at our disposal: | > | > a. Reduce the number of build configurations that we build every commit | > in (e.g. Fedora and LLVM could likely be relegated to a nightly | > build) | > | > b. Reduce the amount of parallelism that we allow CircleCI to employ: | > having access to CI consistently throughout the month is far | > preferable to faster build turnarounds for some of the month | > followed by periods of no CI for the rest. | > | > c. Increase the build time limit; this is $50/month/container | > | > In the short-run we can implement (a) immediately. This of course will | > reduce our coverage, but it looks like our coverage goals may not be | > practical given the resources we have. I've looked into (b) and | > haven't found any indication that CircleCI supports our use-case. If | > this is the case then we will have to simply pursue (c). | > | > I suppose haskell.org may be able to contribute here. However, I | > wonder what other resources could be deployed. | > | > Cheers, | > | > - Ben | > | > | | _______________________________________________ | Ghc-devops-group mailing list | Ghc-devops-group at haskell.org | https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devops-group From ben at well-typed.com Mon Jun 18 13:38:08 2018 From: ben at well-typed.com (Ben Gamari) Date: Mon, 18 Jun 2018 09:38:08 -0400 Subject: [GHC DevOps Group] CircleCI status and budget concerns In-Reply-To: References: <87h8m4rn01.fsf@smart-cactus.org> Message-ID: <87in6gqkox.fsf@smart-cactus.org> Manuel M T Chakravarty writes: > Hi Ben, > > Thanks for bringing this up. > > I agree that (a) is useful short-term fix, but I think, the only > reasonable long-term option is (c). After all, one reason to go with a > hosted CI solution was to be able to solve this kind of problem with > financial support by one of the stakeholders. > > How many containers have we got? (I think, one needs GitHub org admin > access to be able to see that.) > Currently we have 4: CircleCI grants any project one container under their free tier and throws in three more for open-source projects. Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From manuel.chakravarty at tweag.io Tue Jun 19 01:46:41 2018 From: manuel.chakravarty at tweag.io (Manuel M T Chakravarty) Date: Tue, 19 Jun 2018 11:46:41 +1000 Subject: [GHC DevOps Group] CircleCI status and budget concerns In-Reply-To: <87in6gqkox.fsf@smart-cactus.org> References: <87h8m4rn01.fsf@smart-cactus.org> <87in6gqkox.fsf@smart-cactus.org> Message-ID: <42872C28-8A82-4C5C-9795-9D977425FD11@tweag.io> Ok, so we need $200/month (to get a limit extension for four containers). Would any of the institutions represented on this mailing list be able to cover these costs? (The eternal gratitude of the Haskell community would be yours! Oh, and you get a more thoroughly tested compiler!) Cheers, Manuel > Am 18.06.2018 um 23:38 schrieb Ben Gamari : > > Manuel M T Chakravarty writes: > >> Hi Ben, >> >> Thanks for bringing this up. >> >> I agree that (a) is useful short-term fix, but I think, the only >> reasonable long-term option is (c). After all, one reason to go with a >> hosted CI solution was to be able to solve this kind of problem with >> financial support by one of the stakeholders. >> >> How many containers have we got? (I think, one needs GitHub org admin >> access to be able to see that.) >> > Currently we have 4: CircleCI grants any project one container under > their free tier and throws in three more for open-source projects. > > Cheers, > > - Ben > From alex at cloudware.io Tue Jun 19 11:13:16 2018 From: alex at cloudware.io (alex) Date: Tue, 19 Jun 2018 13:13:16 +0200 Subject: [GHC DevOps Group] CircleCI status and budget concerns In-Reply-To: <42872C28-8A82-4C5C-9795-9D977425FD11@tweag.io> References: <87h8m4rn01.fsf@smart-cactus.org> <87in6gqkox.fsf@smart-cactus.org> <42872C28-8A82-4C5C-9795-9D977425FD11@tweag.io> Message-ID: > Would any of the institutions represented on this mailing list be able to cover these costs? I thought Google was already contributing with GCE instances, or is it not the case anymore? On Tue, 19 Jun 2018, 03:46 Manuel M T Chakravarty, < manuel.chakravarty at tweag.io> wrote: > Ok, so we need $200/month (to get a limit extension for four containers). > > Would any of the institutions represented on this mailing list be able to > cover these costs? (The eternal gratitude of the Haskell community would be > yours! Oh, and you get a more thoroughly tested compiler!) > > Cheers, > Manuel > > > Am 18.06.2018 um 23:38 schrieb Ben Gamari : > > > > Manuel M T Chakravarty writes: > > > >> Hi Ben, > >> > >> Thanks for bringing this up. > >> > >> I agree that (a) is useful short-term fix, but I think, the only > >> reasonable long-term option is (c). After all, one reason to go with a > >> hosted CI solution was to be able to solve this kind of problem with > >> financial support by one of the stakeholders. > >> > >> How many containers have we got? (I think, one needs GitHub org admin > >> access to be able to see that.) > >> > > Currently we have 4: CircleCI grants any project one container under > > their free tier and throws in three more for open-source projects. > > > > Cheers, > > > > - Ben > > > > _______________________________________________ > Ghc-devops-group mailing list > Ghc-devops-group at haskell.org > https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devops-group > -------------- next part -------------- An HTML attachment was scrubbed... URL: From alex at centromere.net Tue Jun 19 11:56:05 2018 From: alex at centromere.net (Alex) Date: Tue, 19 Jun 2018 07:56:05 -0400 Subject: [GHC DevOps Group] CircleCI status and budget concerns In-Reply-To: <42872C28-8A82-4C5C-9795-9D977425FD11@tweag.io> References: <87h8m4rn01.fsf@smart-cactus.org> <87in6gqkox.fsf@smart-cactus.org> <42872C28-8A82-4C5C-9795-9D977425FD11@tweag.io> Message-ID: <20180619075605.17b10cf9@centromere.net> On Tue, 19 Jun 2018 11:46:41 +1000 Manuel M T Chakravarty wrote: > Ok, so we need $200/month (to get a limit extension for four > containers). > > Would any of the institutions represented on this mailing list be > able to cover these costs? (The eternal gratitude of the Haskell > community would be yours! Oh, and you get a more thoroughly tested > compiler!) > Would you accept donations from individuals? -- Alex From manuel.chakravarty at tweag.io Wed Jun 20 00:16:16 2018 From: manuel.chakravarty at tweag.io (Manuel M T Chakravarty) Date: Wed, 20 Jun 2018 10:16:16 +1000 Subject: [GHC DevOps Group] CircleCI status and budget concerns In-Reply-To: <20180619075605.17b10cf9@centromere.net> References: <87h8m4rn01.fsf@smart-cactus.org> <87in6gqkox.fsf@smart-cactus.org> <42872C28-8A82-4C5C-9795-9D977425FD11@tweag.io> <20180619075605.17b10cf9@centromere.net> Message-ID: <6EF2BD76-7024-4C5F-93B8-BF064BE99322@tweag.io> Thank you for the offer, but we are currently not set up for this (i.e., it would require a legal entity that can accept donations). Cheers, Manuel > Am 19.06.2018 um 21:56 schrieb Alex : > > On Tue, 19 Jun 2018 11:46:41 +1000 > Manuel M T Chakravarty wrote: > >> Ok, so we need $200/month (to get a limit extension for four >> containers). >> >> Would any of the institutions represented on this mailing list be >> able to cover these costs? (The eternal gratitude of the Haskell >> community would be yours! Oh, and you get a more thoroughly tested >> compiler!) >> > > Would you accept donations from individuals? > > -- > Alex > _______________________________________________ > Ghc-devops-group mailing list > Ghc-devops-group at haskell.org > https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devops-group From gershomb at gmail.com Wed Jun 20 00:26:16 2018 From: gershomb at gmail.com (Gershom B) Date: Tue, 19 Jun 2018 20:26:16 -0400 Subject: [GHC DevOps Group] CircleCI status and budget concerns In-Reply-To: <6EF2BD76-7024-4C5F-93B8-BF064BE99322@tweag.io> References: <87h8m4rn01.fsf@smart-cactus.org> <87in6gqkox.fsf@smart-cactus.org> <42872C28-8A82-4C5C-9795-9D977425FD11@tweag.io> <20180619075605.17b10cf9@centromere.net> <6EF2BD76-7024-4C5F-93B8-BF064BE99322@tweag.io> Message-ID: On June 19, 2018 at 8:16:37 PM, Manuel M T Chakravarty ( manuel.chakravarty at tweag.io) wrote: Thank you for the offer, but we are currently not set up for this (i.e., it would require a legal entity that can accept donations). We do have such a legal entity. That is the purpose of Haskell.org — it is a registered 501c(3) nonprofit which is equipped to handle donations, including earmarked ones, and through which we can organize and manage contributions in a uniform way, using them to fund essential haskell infrastructure — including, e.g. CI systems. We already have a standing offer that has not been acted on yet, to contribute up to I think 50$/mo for CI stuff from haskell.org. We could have some discussion to bump that somewhat, and along with that we certainly could also be a central clearing-house for other earmarked payments that sum up to the total from either individual _or_ institutional contributors. I would not like this offer to pre-empt institutional support from the companies represented in the devops group — after all, a key part of the purpose of this group was to organize and procure support from commercial groups that make use of haskell, and substituting crowdfunding for that won’t get us to that end. However, I do want to be clear that the mechanisms do exist for handling individual contributions, and we can seek a solution that combines that with more reliable long-term institutional commitments to funding. Cheers, Gershom On June 19, 2018 at 8:16:37 PM, Manuel M T Chakravarty ( manuel.chakravarty at tweag.io) wrote: Thank you for the offer, but we are currently not set up for this (i.e., it would require a legal entity that can accept donations). Cheers, Manuel > Am 19.06.2018 um 21:56 schrieb Alex : > > On Tue, 19 Jun 2018 11:46:41 +1000 > Manuel M T Chakravarty wrote: > >> Ok, so we need $200/month (to get a limit extension for four >> containers). >> >> Would any of the institutions represented on this mailing list be >> able to cover these costs? (The eternal gratitude of the Haskell >> community would be yours! Oh, and you get a more thoroughly tested >> compiler!) >> > > Would you accept donations from individuals? > > -- > Alex > _______________________________________________ > Ghc-devops-group mailing list > Ghc-devops-group at haskell.org > https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devops-group _______________________________________________ Ghc-devops-group mailing list Ghc-devops-group at haskell.org https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devops-group -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben at well-typed.com Wed Jun 20 01:37:10 2018 From: ben at well-typed.com (Ben Gamari) Date: Tue, 19 Jun 2018 21:37:10 -0400 Subject: [GHC DevOps Group] CircleCI status and budget concerns In-Reply-To: References: <87h8m4rn01.fsf@smart-cactus.org> <87in6gqkox.fsf@smart-cactus.org> <42872C28-8A82-4C5C-9795-9D977425FD11@tweag.io> Message-ID: <871sd25jcz.fsf@smart-cactus.org> alex writes: >> Would any of the institutions represented on this mailing list be able to > cover these costs? > > I thought Google was already contributing with GCE instances, or is it not > the case anymore? > Google is indeed contributing GCE instances for our Appveyor infrastructure. The CircleCI offer is new. Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From ben at well-typed.com Wed Jun 20 01:38:32 2018 From: ben at well-typed.com (Ben Gamari) Date: Tue, 19 Jun 2018 21:38:32 -0400 Subject: [GHC DevOps Group] CircleCI status and budget concerns In-Reply-To: <6EF2BD76-7024-4C5F-93B8-BF064BE99322@tweag.io> References: <87h8m4rn01.fsf@smart-cactus.org> <87in6gqkox.fsf@smart-cactus.org> <42872C28-8A82-4C5C-9795-9D977425FD11@tweag.io> <20180619075605.17b10cf9@centromere.net> <6EF2BD76-7024-4C5F-93B8-BF064BE99322@tweag.io> Message-ID: <87wouu44q2.fsf@smart-cactus.org> Manuel M T Chakravarty writes: > Thank you for the offer, but we are currently not set up for this > (i.e., it would require a legal entity that can accept donations). > I agree with Gershom and was going to suggest that we use haskell.org for exactly this. Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From manuel.chakravarty at tweag.io Wed Jun 20 01:53:06 2018 From: manuel.chakravarty at tweag.io (Manuel M T Chakravarty) Date: Wed, 20 Jun 2018 11:53:06 +1000 Subject: [GHC DevOps Group] CircleCI status and budget concerns In-Reply-To: References: <87h8m4rn01.fsf@smart-cactus.org> <87in6gqkox.fsf@smart-cactus.org> <42872C28-8A82-4C5C-9795-9D977425FD11@tweag.io> <20180619075605.17b10cf9@centromere.net> <6EF2BD76-7024-4C5F-93B8-BF064BE99322@tweag.io> Message-ID: <61276905-1AFE-4ACF-8260-D93C2B048F0B@tweag.io> Thanks, Gershom. It’s good to know that we have got this option. The immediate issue with CircleCI is resolved as Google X have generously offered to fund this for a year. They now help us out with both AppVeyor and CircleCI. Thank you, Google X! Cheers, Manuel > 20.06.2018 10:26 Gershom B : > > On June 19, 2018 at 8:16:37 PM, Manuel M T Chakravarty (manuel.chakravarty at tweag.io ) wrote: > >> Thank you for the offer, but we are currently not set up for this (i.e., it would require a legal entity that can accept donations). > > > > We do have such a legal entity. That is the purpose of Haskell.org — it is a registered 501c(3) nonprofit which is equipped to handle donations, including earmarked ones, and through which we can organize and manage contributions in a uniform way, using them to fund essential haskell infrastructure — including, e.g. CI systems. > > We already have a standing offer that has not been acted on yet, to contribute up to I think 50$/mo for CI stuff from haskell.org . We could have some discussion to bump that somewhat, and along with that we certainly could also be a central clearing-house for other earmarked payments that sum up to the total from either individual _or_ institutional contributors. > > I would not like this offer to pre-empt institutional support from the companies represented in the devops group — after all, a key part of the purpose of this group was to organize and procure support from commercial groups that make use of haskell, and substituting crowdfunding for that won’t get us to that end. However, I do want to be clear that the mechanisms do exist for handling individual contributions, and we can seek a solution that combines that with more reliable long-term institutional commitments to funding. > > Cheers, > Gershom > > On June 19, 2018 at 8:16:37 PM, Manuel M T Chakravarty (manuel.chakravarty at tweag.io ) wrote: > >> Thank you for the offer, but we are currently not set up for this (i.e., it would require a legal entity that can accept donations). >> >> Cheers, >> Manuel >> >> > Am 19.06.2018 um 21:56 schrieb Alex >: >> > >> > On Tue, 19 Jun 2018 11:46:41 +1000 >> > Manuel M T Chakravarty > wrote: >> > >> >> Ok, so we need $200/month (to get a limit extension for four >> >> containers). >> >> >> >> Would any of the institutions represented on this mailing list be >> >> able to cover these costs? (The eternal gratitude of the Haskell >> >> community would be yours! Oh, and you get a more thoroughly tested >> >> compiler!) >> >> >> > >> > Would you accept donations from individuals? >> > >> > -- >> > Alex >> > _______________________________________________ >> > Ghc-devops-group mailing list >> > Ghc-devops-group at haskell.org >> > https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devops-group >> >> _______________________________________________ >> Ghc-devops-group mailing list >> Ghc-devops-group at haskell.org >> https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devops-group > _______________________________________________ > Ghc-devops-group mailing list > Ghc-devops-group at haskell.org > https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devops-group -------------- next part -------------- An HTML attachment was scrubbed... URL: