From manuel.chakravarty at tweag.io Thu Dec 6 16:05:16 2018 From: manuel.chakravarty at tweag.io (Manuel Chakravarty) Date: Thu, 6 Dec 2018 17:05:16 +0100 Subject: [GHC DevOps Group] Guidelines Message-ID: <9AF21783-FA63-4D16-AD63-D4B024378885@tweag.io> Dear DevOps Group, The GHC Steering Committee has recently decided to adopt a set of ”Guidelines for Respectful Communication” due to ongoing concern about the standards of discourse in the Haskell community: https://github.com/ghc-proposals/ghc-proposals/blob/master/GRC.rst I’ll leave it to Simon PJ’s excellent explanation to lay out the detailed motivation: > As many of you will know, I’ve been concerned for several years about the standards of discourse in the Haskell community. I think things have improved since the period that drove me to write my Respect email , but it’s far from secure. > > I have thought about trying to introduce a code of conduct for the Haskell community, but (a) getting agreement about such a thing would be hard, (b) we have no credible enforcement mechanism and (c) some people disagree that it would even be helpful. > > I have also considered publishing a kind of personal code of conduct, describing the standards to which I hold myself (doubtless all too fallibly). I was concerned that this would over-personalise the issue. > > In the light of all this, following discussion at ICFP, the GHC Steering Committee has decided to adopt these "Guidelines for Respectful Communication”. > > as a middle way. We are adopting it for ourselves, not imposing it on others. But in doing so we hope to give a clear lead to the community about the standards that we espouse for our professional dialogue. > > We deliberately decided to use the term “guidelines for respectful communication”, rather than “code of conduct”, for reasons that Stallman lays out in his recently-released GNU Kind Communications Guidelines: https://lwn.net/Articles/769167/ In this context, it is important to note that the intention for these guidelines is to apply to us (the DevOps group members) with the intention of setting a good example and not to police the broader community, which we do not have the mandate to do anyway. Personally, I am strongly in favour for us to follow the example of the GHC Steering Committee and to adopt the same rules, in order to project a uniform picture from all GHC committees. Cheers, Manuel From ben at well-typed.com Wed Dec 12 23:21:27 2018 From: ben at well-typed.com (Ben Gamari) Date: Wed, 12 Dec 2018 18:21:27 -0500 Subject: [GHC DevOps Group] CircleCI status Message-ID: <8736r2tjpm.fsf@smart-cactus.org> TL;DR. CircleCI's free tier will no longer include the large instances used by GHC CI starting next week. Hello everyone, Prompted by a rather surprisingly short-notice email from CircleCI I just got off of the phone with them to discuss a change in their pricing model. This new model, which they will be instituting later this week, will users of medium+ and larger instances will no longer be covered by the free tier of service which we currently use. Currently we are using CircleCI for only a fraction of what we would like to validate: * commit validation (which we have been checking manually while we transition to GitLab) in a handful of configuration * nightly intensive validation * release binary distributions (but don't yet produce all distributions via CI) Ultimately we would like to also introduce merge request validation, which will add a large volume of builds once implemented later this month. However, even our current usage totals to around 2.5 million CircleCI credits. Up to this point we have been covered by CircleCI's free tier service. However, under the new pricing model the extra-large instances which we use are no longer covered in the free tier. Note that medium instances are still covered by the free tier service. However, the GHC build on such an instance does not fit within CircleCI's five hour build-time limit. It seems we have a few options: a. Pay for our credits. The 2.5 million credits we used last month would cost approximately 2,400 USD/month. I asked CircleCI last summer and when I was on the phone with them today and they will not offer us a reduced price. b. Find another solution. Thankfully, the move to GitLab has freed us from any particular CI provider. Happily, our CI setup is nearly fully Dockerised at this point, so moving is quite straightforward. With the move to GitLab, we have no need for the provisioning and queue management functionality that CircleCI provides. Finally, I find the 2,400 USD/month number to be extremely high given the service we are getting. By comparison: * The machine sitting next to my desk cost ~$2,500, a couple hundred USD/year in electricity and yet finishes six builds in the time it takes a CircleCI XL instance to finish one. * A machine comparable to mine from Packet.net is only 700 USD/month * A EC2 instance comparable to the CircleCI XL would only amount to a few hundred USD/month. * A dedicated Hetzner box comparable to the XL is ~40 EUR/month In my opinion, any of these solutions would be a better use of our limited resources. Regardless, I have convinced CircleCI to extend us a one-week trial period allowing us a bit of breathing room. If necessary we can pay for CircleCI on a month-by-month basis if we need additional time. - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From gnezdo at google.com Thu Dec 13 06:09:24 2018 From: gnezdo at google.com (Greg Steuck (Sh-toy-k)) Date: Wed, 12 Dec 2018 22:09:24 -0800 Subject: [GHC DevOps Group] CircleCI status In-Reply-To: <8736r2tjpm.fsf@smart-cactus.org> References: <8736r2tjpm.fsf@smart-cactus.org> Message-ID: Hi Ben, We are only using ~$400/month in GHC VM donation GCE project (Windows license claiming the lions share). If you need more compute, there's plenty of headroom left. Thanks Greg On Wed, Dec 12, 2018 at 3:21 PM Ben Gamari wrote: > TL;DR. CircleCI's free tier will no longer include the large instances > used by GHC CI starting next week. > > > Hello everyone, > > Prompted by a rather surprisingly short-notice email from CircleCI I > just got off of the phone with them to discuss a change in their pricing > model. This new model, which they will be instituting later this week, > will users of medium+ and larger instances will no longer be covered > by the free tier of service which we currently use. > > Currently we are using CircleCI for only a fraction of what we would > like to validate: > > * commit validation (which we have been checking manually while we > transition to GitLab) in a handful of configuration > * nightly intensive validation > * release binary distributions (but don't yet produce all distributions > via CI) > > Ultimately we would like to also introduce merge request validation, which > will add a large volume of builds once implemented later this month. > > However, even our current usage totals to around 2.5 million CircleCI > credits. Up to this point we have been covered by CircleCI's free tier > service. However, under the new pricing model the extra-large instances > which we use are no longer covered in the free tier. > > Note that medium instances are still covered by the free tier service. > However, the GHC build on such an instance does not fit within > CircleCI's five hour build-time limit. > > It seems we have a few options: > > a. Pay for our credits. The 2.5 million credits we used last month > would cost approximately 2,400 USD/month. I asked CircleCI last > summer and when I was on the phone with them today and they will not > offer us a reduced price. > > b. Find another solution. Thankfully, the move to GitLab has freed us > from any particular CI provider. > > Happily, our CI setup is nearly fully Dockerised at this point, so > moving is quite straightforward. With the move to GitLab, we have no > need for the provisioning and queue management functionality that > CircleCI provides. > > Finally, I find the 2,400 USD/month number to be extremely high given > the service we are getting. By comparison: > > * The machine sitting next to my desk cost ~$2,500, a couple hundred > USD/year in electricity and yet finishes six builds in the time it > takes a CircleCI XL instance to finish one. > > * A machine comparable to mine from Packet.net is only 700 USD/month > > * A EC2 instance comparable to the CircleCI XL would only amount to a > few hundred USD/month. > > * A dedicated Hetzner box comparable to the XL is ~40 EUR/month > > In my opinion, any of these solutions would be a better use of our > limited resources. > > Regardless, I have convinced CircleCI to extend us a one-week trial > period allowing us a bit of breathing room. If necessary we can pay for > CircleCI on a month-by-month basis if we need additional time. > > - Ben > _______________________________________________ > Ghc-devops-group mailing list > Ghc-devops-group at haskell.org > https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devops-group > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4843 bytes Desc: S/MIME Cryptographic Signature URL: From ben at well-typed.com Fri Dec 14 02:12:28 2018 From: ben at well-typed.com (Ben Gamari) Date: Thu, 13 Dec 2018 21:12:28 -0500 Subject: [GHC DevOps Group] CircleCI status In-Reply-To: References: <8736r2tjpm.fsf@smart-cactus.org> Message-ID: <87tvjgsvp4.fsf@smart-cactus.org> "Greg Steuck (Sh-toy-k)" writes: > Hi Ben, > > We are only using ~$400/month in GHC VM donation GCE project (Windows > license claiming the lions share). If you need more compute, there's plenty > of headroom left. > Yes, thank you Greg! That would be very helpful and certainly enables us to move ahead with GitLab. Can you remind me what the upper bound on our usage is? Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From ndmitchell at gmail.com Fri Dec 14 11:41:12 2018 From: ndmitchell at gmail.com (Neil Mitchell) Date: Fri, 14 Dec 2018 11:41:12 +0000 Subject: [GHC DevOps Group] CircleCI status In-Reply-To: <87tvjgsvp4.fsf@smart-cactus.org> References: <8736r2tjpm.fsf@smart-cactus.org> <87tvjgsvp4.fsf@smart-cactus.org> Message-ID: FWIW for Hadrian we were considering Azure: https://azure.microsoft.com/en-us/blog/announcing-azure-pipelines-with-unlimited-ci-cd-minutes-for-open-source/ To quote: > Starting today, Azure Pipelines provides unlimited CI/CD minutes and 10 parallel jobs to every open source project for free. All open source projects run on the same infrastructure that our paying customers use. Thanks, Neil On Fri, Dec 14, 2018 at 2:12 AM Ben Gamari wrote: > > "Greg Steuck (Sh-toy-k)" writes: > > > Hi Ben, > > > > We are only using ~$400/month in GHC VM donation GCE project (Windows > > license claiming the lions share). If you need more compute, there's plenty > > of headroom left. > > > Yes, thank you Greg! That would be very helpful and certainly enables us > to move ahead with GitLab. Can you remind me what the upper bound on our > usage is? > > Cheers, > > - Ben > > _______________________________________________ > Ghc-devops-group mailing list > Ghc-devops-group at haskell.org > https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devops-group From ben at well-typed.com Mon Dec 17 05:29:00 2018 From: ben at well-typed.com (Ben Gamari) Date: Mon, 17 Dec 2018 00:29:00 -0500 Subject: [GHC DevOps Group] GitLab migration status Message-ID: <871s6gsovc.fsf@smart-cactus.org> TL;DR. Given somewhat slower-than-expected progress on the Trac import I suggest that we implement a pared-down migration on Tuesday. See "The Plan" below. Hello everyone, Over the last few weeks we have been hard at work preparing the migration to GitLab. Currently the following things are ready: * Hosting of GHC's repositories and those of its mirrors have been prepared. * Continuous integration has been configured for GHC. All-in-all the GitLab migration has been quite timely since we were recently notified by CircleCI of billing changes which will soon make it quite difficult for us to continue using their services (see the thread on ghc-devops for details). Thankfully, moving CI to GitLab has been mostly painless and has even enabled us to introduce testing of platforms which were previously inaccessible to us under CircleCI. * The various linters which previously ran via `arc lint` and gitolite post-receive hooks have been ported to CI jobs. * The Trac ticket migration is looking good although there are still a significant number of details which need to be sorted out. * The Wiki migration is in a similar state. Over the past weeks we have been in constant contact with GitLab's FOSS outreach group, who have been quite helpful in getting the eyes of GitLab employees on the issues affecting our transition. Thanks to especially to David Planella for his help so far. Unfortunately, there is one issue in particular [1] which is currently blocking the Trac migration. From my discussions with GitLab's upstream it sounds like it may be possible for them to prioritize a fix in the short-term. However, our aggressive migration timeline is a fair bit faster than GitLab's development cycle and consequently this certainly won't happen before our planned migration on Tuesday. # The Plan Given what remains to be done in the Trac migration I believe it would be a mistake to move ahead with the full migration as planned. However, in the interest of re-gaining functional continuous integration of patches as soon as possible I propose that we move ahead with moving code review on Tuesday. The plan would be as follows: 1. We setup the final gitlab.haskell.org instance tomorrow; since the Trac migration will not be run will need to create new accounts on instance. 2. We begin officially accepting merge requests on this fresh GitLab instance on Tuesday. At this point gitlab.haskell.org:ghc/ghc will become GHC's official upstream repository. 3. We allow a week of transition time where new Differentials will continue to be accepted via Phabricator. 4. After this transition period we place Phabricator in read-only mode. 5. When we are confident in the Trac migration (likely after the new year) we move ahead with importing tickets and the wiki Previously I was skeptical of any plan that involved running the Trac migration against a live GitLab instance. However, further reflection I believe such a migration is safe and feasible. Moreover, given the constraints set upon us by the impending CircleCI changes, I think this is our best option to ensure continuity of CI. Thoughts? Cheers, - Ben [1] https://gitlab.com/gitlab-org/gitlab-ce/issues/46980 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From marlowsd at gmail.com Mon Dec 17 08:03:25 2018 From: marlowsd at gmail.com (Simon Marlow) Date: Mon, 17 Dec 2018 08:03:25 +0000 Subject: [GHC DevOps Group] GitLab migration status In-Reply-To: <871s6gsovc.fsf@smart-cactus.org> References: <871s6gsovc.fsf@smart-cactus.org> Message-ID: Hi Ben - this sounds good, a couple of questions: - What about the performance issue we noticed last week? - What will happen to Phabricator diffs that are still mid-review? It would be a shame to have to move them to gitlab and interrupt the review trail. Can't we just shut Phabricator to new diffs but keep the possibility of working on existing ones? Cheers Simon On Mon, 17 Dec 2018 at 05:29, Ben Gamari wrote: > TL;DR. Given somewhat slower-than-expected progress on the Trac import I > suggest that we implement a pared-down migration on Tuesday. > See "The Plan" below. > > > Hello everyone, > > Over the last few weeks we have been hard at work preparing the > migration to GitLab. Currently the following things are ready: > > * Hosting of GHC's repositories and those of its mirrors have been > prepared. > > * Continuous integration has been configured for GHC. > > All-in-all the GitLab migration has been quite timely since we were > recently notified by CircleCI of billing changes which will soon make > it quite difficult for us to continue using their services (see the > thread on ghc-devops for details). > > Thankfully, moving CI to GitLab has been mostly painless and has > even enabled us to introduce testing of platforms which were > previously inaccessible to us under CircleCI. > > * The various linters which previously ran via `arc lint` and gitolite > post-receive hooks have been ported to CI jobs. > > * The Trac ticket migration is looking good although there are still a > significant number of details which need to be sorted out. > > * The Wiki migration is in a similar state. > > Over the past weeks we have been in constant contact with GitLab's FOSS > outreach group, who have been quite helpful in getting the eyes of > GitLab employees on the issues affecting our transition. Thanks to > especially to David Planella for his help so far. > > Unfortunately, there is one issue in particular [1] which is currently > blocking the Trac migration. From my discussions with GitLab's upstream > it sounds like it may be possible for them to prioritize a fix in the > short-term. However, our aggressive migration timeline is a fair bit > faster than GitLab's development cycle and consequently this certainly > won't happen before our planned migration on Tuesday. > > > # The Plan > > Given what remains to be done in the Trac migration I believe it would > be a mistake to move ahead with the full migration as planned. However, > in the interest of re-gaining functional continuous integration of > patches as soon as possible I propose that we move ahead with moving > code review on Tuesday. > > The plan would be as follows: > > 1. We setup the final gitlab.haskell.org instance tomorrow; since the > Trac migration will not be run will need to create new accounts on > instance. > > 2. We begin officially accepting merge requests on this fresh GitLab > instance on Tuesday. At this point gitlab.haskell.org:ghc/ghc will > become GHC's official upstream repository. > > 3. We allow a week of transition time where new Differentials will > continue to be accepted via Phabricator. > > 4. After this transition period we place Phabricator in read-only mode. > > 5. When we are confident in the Trac migration (likely after the new > year) we move ahead with importing tickets and the wiki > > Previously I was skeptical of any plan that involved running the Trac > migration against a live GitLab instance. However, further reflection > I believe such a migration is safe and feasible. Moreover, given the > constraints set upon us by the impending CircleCI changes, I think this > is our best option to ensure continuity of CI. > > Thoughts? > > Cheers, > > - Ben > > > [1] https://gitlab.com/gitlab-org/gitlab-ce/issues/46980 > _______________________________________________ > Ghc-devops-group mailing list > Ghc-devops-group at haskell.org > https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devops-group > -------------- next part -------------- An HTML attachment was scrubbed... URL: From simon.jakobi at googlemail.com Mon Dec 17 08:17:18 2018 From: simon.jakobi at googlemail.com (Simon Jakobi) Date: Mon, 17 Dec 2018 09:17:18 +0100 Subject: [GHC DevOps Group] GitLab migration status In-Reply-To: <871s6gsovc.fsf@smart-cactus.org> References: <871s6gsovc.fsf@smart-cactus.org> Message-ID: Hi Ben, in my experience Gitlab has been extremely slow at showing commit diffs to the point that it gives up and returns a 502: https://gitlab.staging.haskell.org/ghc/ghc/issues/15944 Is this possibly related to any resource constraints on our instance? Cheers, Simon Am Mo., 17. Dez. 2018 um 06:29 Uhr schrieb Ben Gamari : > TL;DR. Given somewhat slower-than-expected progress on the Trac import I > suggest that we implement a pared-down migration on Tuesday. > See "The Plan" below. > > > Hello everyone, > > Over the last few weeks we have been hard at work preparing the > migration to GitLab. Currently the following things are ready: > > * Hosting of GHC's repositories and those of its mirrors have been > prepared. > > * Continuous integration has been configured for GHC. > > All-in-all the GitLab migration has been quite timely since we were > recently notified by CircleCI of billing changes which will soon make > it quite difficult for us to continue using their services (see the > thread on ghc-devops for details). > > Thankfully, moving CI to GitLab has been mostly painless and has > even enabled us to introduce testing of platforms which were > previously inaccessible to us under CircleCI. > > * The various linters which previously ran via `arc lint` and gitolite > post-receive hooks have been ported to CI jobs. > > * The Trac ticket migration is looking good although there are still a > significant number of details which need to be sorted out. > > * The Wiki migration is in a similar state. > > Over the past weeks we have been in constant contact with GitLab's FOSS > outreach group, who have been quite helpful in getting the eyes of > GitLab employees on the issues affecting our transition. Thanks to > especially to David Planella for his help so far. > > Unfortunately, there is one issue in particular [1] which is currently > blocking the Trac migration. From my discussions with GitLab's upstream > it sounds like it may be possible for them to prioritize a fix in the > short-term. However, our aggressive migration timeline is a fair bit > faster than GitLab's development cycle and consequently this certainly > won't happen before our planned migration on Tuesday. > > > # The Plan > > Given what remains to be done in the Trac migration I believe it would > be a mistake to move ahead with the full migration as planned. However, > in the interest of re-gaining functional continuous integration of > patches as soon as possible I propose that we move ahead with moving > code review on Tuesday. > > The plan would be as follows: > > 1. We setup the final gitlab.haskell.org instance tomorrow; since the > Trac migration will not be run will need to create new accounts on > instance. > > 2. We begin officially accepting merge requests on this fresh GitLab > instance on Tuesday. At this point gitlab.haskell.org:ghc/ghc will > become GHC's official upstream repository. > > 3. We allow a week of transition time where new Differentials will > continue to be accepted via Phabricator. > > 4. After this transition period we place Phabricator in read-only mode. > > 5. When we are confident in the Trac migration (likely after the new > year) we move ahead with importing tickets and the wiki > > Previously I was skeptical of any plan that involved running the Trac > migration against a live GitLab instance. However, further reflection > I believe such a migration is safe and feasible. Moreover, given the > constraints set upon us by the impending CircleCI changes, I think this > is our best option to ensure continuity of CI. > > Thoughts? > > Cheers, > > - Ben > > > [1] https://gitlab.com/gitlab-org/gitlab-ce/issues/46980 > _______________________________________________ > Ghc-devops-group mailing list > Ghc-devops-group at haskell.org > https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devops-group > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben at well-typed.com Mon Dec 17 15:14:31 2018 From: ben at well-typed.com (Ben Gamari) Date: Mon, 17 Dec 2018 10:14:31 -0500 Subject: [GHC DevOps Group] GitLab migration status In-Reply-To: References: <871s6gsovc.fsf@smart-cactus.org> Message-ID: <87woo8qj71.fsf@smart-cactus.org> Simon Jakobi via ghc-devs writes: > Hi Ben, > > in my experience Gitlab has been extremely slow at showing commit diffs to > the point that it gives up and returns a 502: > https://gitlab.staging.haskell.org/ghc/ghc/issues/15944 > > Is this possibly related to any resource constraints on our instance? > Yes, I have also noticed the general slowness of our instance. Last week I identified the cause as being the fact that it is being hosted on an AArch64 box, which apparently isn't a configuration that is very well supported by the Ruby interpreter's optimised implementation. The same instance running on x86_64 is much more responsive [1]. Cheers, - Ben [1] https://gitlab.staging2.haskell.org/ghc/ghc/merge_requests/11 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From ben at well-typed.com Mon Dec 17 15:52:42 2018 From: ben at well-typed.com (Ben Gamari) Date: Mon, 17 Dec 2018 10:52:42 -0500 Subject: [GHC DevOps Group] GitLab migration status In-Reply-To: References: <871s6gsovc.fsf@smart-cactus.org> Message-ID: <87sgywqhfb.fsf@smart-cactus.org> Simon Marlow writes: > Hi Ben - this sounds good, a couple of questions: > > - What about the performance issue we noticed last week? I identified the problem as being the ARM environment which the staging instance is running on. The final instance will run on x86_64. > - What will happen to Phabricator diffs that are still mid-review? It would > be a shame to have to move them to gitlab and interrupt the review trail. > Can't we just shut Phabricator to new diffs but keep the possibility of > working on existing ones? > Of course. That is perhaps a better path forward. Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From carter.schonwald at gmail.com Tue Dec 18 18:33:34 2018 From: carter.schonwald at gmail.com (Carter Schonwald) Date: Tue, 18 Dec 2018 13:33:34 -0500 Subject: [GHC DevOps Group] GitLab migration status In-Reply-To: References: <871s6gsovc.fsf@smart-cactus.org> Message-ID: Hey simon! I actually did some digging. And it looks like currently gitlab basically does a git diff of the two branches and does that computation on every page reload. 1). There currently is no caching —- some of the folks on the Haskell infra team could help perhaps but they need to be given access to the setup for that to happen. 2) the complexity of the git diff is linear in the commit history length between the two branches being compared Some of these things could be mitigated easily, but I spoke with one of the folks on the Haskell infra team and it sounds like ultimately either patching the gitlab gitaly server or providing a more efficient api compatible replacement are what would be needed. On Mon, Dec 17, 2018 at 3:18 AM Simon Jakobi via ghc-devs < ghc-devs at haskell.org> wrote: > Hi Ben, > > in my experience Gitlab has been extremely slow at showing commit diffs to > the point that it gives up and returns a 502: > https://gitlab.staging.haskell.org/ghc/ghc/issues/15944 > > Is this possibly related to any resource constraints on our instance? > > Cheers, > Simon > > Am Mo., 17. Dez. 2018 um 06:29 Uhr schrieb Ben Gamari >: > >> TL;DR. Given somewhat slower-than-expected progress on the Trac import I >> suggest that we implement a pared-down migration on Tuesday. >> See "The Plan" below. >> >> >> Hello everyone, >> >> Over the last few weeks we have been hard at work preparing the >> migration to GitLab. Currently the following things are ready: >> >> * Hosting of GHC's repositories and those of its mirrors have been >> prepared. >> >> * Continuous integration has been configured for GHC. >> >> All-in-all the GitLab migration has been quite timely since we were >> recently notified by CircleCI of billing changes which will soon make >> it quite difficult for us to continue using their services (see the >> thread on ghc-devops for details). >> >> Thankfully, moving CI to GitLab has been mostly painless and has >> even enabled us to introduce testing of platforms which were >> previously inaccessible to us under CircleCI. >> >> * The various linters which previously ran via `arc lint` and gitolite >> post-receive hooks have been ported to CI jobs. >> >> * The Trac ticket migration is looking good although there are still a >> significant number of details which need to be sorted out. >> >> * The Wiki migration is in a similar state. >> >> Over the past weeks we have been in constant contact with GitLab's FOSS >> outreach group, who have been quite helpful in getting the eyes of >> GitLab employees on the issues affecting our transition. Thanks to >> especially to David Planella for his help so far. >> >> Unfortunately, there is one issue in particular [1] which is currently >> blocking the Trac migration. From my discussions with GitLab's upstream >> it sounds like it may be possible for them to prioritize a fix in the >> short-term. However, our aggressive migration timeline is a fair bit >> faster than GitLab's development cycle and consequently this certainly >> won't happen before our planned migration on Tuesday. >> >> >> # The Plan >> >> Given what remains to be done in the Trac migration I believe it would >> be a mistake to move ahead with the full migration as planned. However, >> in the interest of re-gaining functional continuous integration of >> patches as soon as possible I propose that we move ahead with moving >> code review on Tuesday. >> >> The plan would be as follows: >> >> 1. We setup the final gitlab.haskell.org instance tomorrow; since the >> Trac migration will not be run will need to create new accounts on >> instance. >> >> 2. We begin officially accepting merge requests on this fresh GitLab >> instance on Tuesday. At this point gitlab.haskell.org:ghc/ghc will >> become GHC's official upstream repository. >> >> 3. We allow a week of transition time where new Differentials will >> continue to be accepted via Phabricator. >> >> 4. After this transition period we place Phabricator in read-only mode. >> >> 5. When we are confident in the Trac migration (likely after the new >> year) we move ahead with importing tickets and the wiki >> >> Previously I was skeptical of any plan that involved running the Trac >> migration against a live GitLab instance. However, further reflection >> I believe such a migration is safe and feasible. Moreover, given the >> constraints set upon us by the impending CircleCI changes, I think this >> is our best option to ensure continuity of CI. >> >> Thoughts? >> >> Cheers, >> >> - Ben >> >> >> [1] https://gitlab.com/gitlab-org/gitlab-ce/issues/46980 >> _______________________________________________ >> Ghc-devops-group mailing list >> Ghc-devops-group at haskell.org >> https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devops-group >> > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rae at cs.brynmawr.edu Tue Dec 18 03:25:06 2018 From: rae at cs.brynmawr.edu (Richard Eisenberg) Date: Mon, 17 Dec 2018 22:25:06 -0500 Subject: [GHC DevOps Group] GitLab migration status In-Reply-To: <87sgywqhfb.fsf@smart-cactus.org> References: <871s6gsovc.fsf@smart-cactus.org> <87sgywqhfb.fsf@smart-cactus.org> Message-ID: <92A44AB2-ACCB-4A24-A17D-FBC4CE305B06@cs.brynmawr.edu> > On Dec 17, 2018, at 10:52 AM, Ben Gamari wrote: > >> >> - What will happen to Phabricator diffs that are still mid-review? It would >> be a shame to have to move them to gitlab and interrupt the review trail. >> Can't we just shut Phabricator to new diffs but keep the possibility of >> working on existing ones? >> > Of course. That is perhaps a better path forward. I second this notion, that we stop accepting new diffs on Phab but continue to work with existing diffs until they are merged or abandoned (depending on how resource-intensive it is to do this). Thanks, Richard -------------- next part -------------- An HTML attachment was scrubbed... URL: From gnezdo at google.com Wed Dec 19 20:49:42 2018 From: gnezdo at google.com (Greg Steuck (Sh-toy-k)) Date: Wed, 19 Dec 2018 12:49:42 -0800 Subject: [GHC DevOps Group] CircleCI status In-Reply-To: <87tvjgsvp4.fsf@smart-cactus.org> References: <8736r2tjpm.fsf@smart-cactus.org> <87tvjgsvp4.fsf@smart-cactus.org> Message-ID: I checked and we can go 10x what we have been using up to now. We'll have a spending cap of $4500/month set up. If you can get away from Windows, the licensing fees will stop eating into this budget. Thanks Greg On Thu, Dec 13, 2018 at 6:12 PM Ben Gamari wrote: > "Greg Steuck (Sh-toy-k)" writes: > > > Hi Ben, > > > > We are only using ~$400/month in GHC VM donation GCE project (Windows > > license claiming the lions share). If you need more compute, there's > plenty > > of headroom left. > > > Yes, thank you Greg! That would be very helpful and certainly enables us > to move ahead with GitLab. Can you remind me what the upper bound on our > usage is? > > Cheers, > > - Ben > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4843 bytes Desc: S/MIME Cryptographic Signature URL: From ben at well-typed.com Sat Dec 22 05:12:46 2018 From: ben at well-typed.com (Ben Gamari) Date: Sat, 22 Dec 2018 00:12:46 -0500 Subject: [GHC DevOps Group] GitLab transition status Message-ID: <87a7kyp2k5.fsf@smart-cactus.org> Hello everyone, Over the past week we have been busily pushing ahead with the GitLab transition. While the original plan was to switch the upstream repository to the GitLab on last Tuesday, I have been too preoccupied pushing through the last mile of continuous integration stabilization to write the documentation that would make such a switch possible. Consequently, GitLab's ghc/ghc repository is still mirroring git.haskell.org. Continuous integration via GitLab pipelines is now very nearly green [1]. Once this happens I will send an email with final migration instructions and disable mirroring, signalling that the GitLab repository is the official upstream. After we migrate the upstream all code entering `master` will be merged via merge request, ensuring that no further breakage in CI. It would be greatly appreciated if those with commit rights could refrain from pushing unnecessary changes until the migration is finalized. I have had to fix a few breakages this week, each of which costs a few hours in effort and build time. Cheers, - Ben [1] https://gitlab.haskell.org/ghc/ghc/pipelines/444 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From ben at well-typed.com Sat Dec 22 15:44:38 2018 From: ben at well-typed.com (Ben Gamari) Date: Sat, 22 Dec 2018 10:44:38 -0500 Subject: [GHC DevOps Group] CircleCI status In-Reply-To: References: <8736r2tjpm.fsf@smart-cactus.org> <87tvjgsvp4.fsf@smart-cactus.org> Message-ID: <87woo1o9b2.fsf@smart-cactus.org> "Greg Steuck (Sh-toy-k)" writes: > I checked and we can go 10x what we have been using up to now. We'll have a > spending cap of $4500/month set up. > > If you can get away from Windows, the licensing fees will stop eating into > this budget. > Indeed. Currently Windows is eating a disproportionate amount of our budget due to a perfect storm of considerations: a. Windows licensing fees, meaning Windows instances cost two-to-three times what we pay per-core for Linux instances b. the Windows linker is slow as mollasses, meaning we need lots of cores to keep up with our build volume Build times are currently over three hours on the current four-core machine we current have on GCE with over half of this being spent in the testsuite where almost all time is spent in ld. I have made some inquiries to see whether there is anything we can do about this. In the short term I intend on increasing the size of our instance to try to keep up with demand a bit better. Thanks again for your help! Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From gnezdo at google.com Sat Dec 22 17:33:11 2018 From: gnezdo at google.com (Greg Steuck (Sh-toy-k)) Date: Sat, 22 Dec 2018 09:33:11 -0800 Subject: [GHC DevOps Group] CircleCI status In-Reply-To: <87woo1o9b2.fsf@smart-cactus.org> References: <8736r2tjpm.fsf@smart-cactus.org> <87tvjgsvp4.fsf@smart-cactus.org> <87woo1o9b2.fsf@smart-cactus.org> Message-ID: FWIW, lld seems pretty well supported on Windows . We have some evidence that lld is faster than gold and ld.bfd on Linux. Thanks Greg On Sat, Dec 22, 2018 at 7:44 AM Ben Gamari wrote: > "Greg Steuck (Sh-toy-k)" writes: > > > I checked and we can go 10x what we have been using up to now. We'll > have a > > spending cap of $4500/month set up. > > > > If you can get away from Windows, the licensing fees will stop eating > into > > this budget. > > > Indeed. Currently Windows is eating a disproportionate amount of our > budget due to a perfect storm of considerations: > > a. Windows licensing fees, meaning Windows instances cost two-to-three > times what we pay per-core for Linux instances > > b. the Windows linker is slow as mollasses, meaning we need lots of > cores to keep up with our build volume > > Build times are currently over three hours on the current four-core > machine we current have on GCE with over half of this being spent in the > testsuite where almost all time is spent in ld. I have made some > inquiries to see whether there is anything we can do about this. > > In the short term I intend on increasing the size of our instance to try > to keep up with demand a bit better. > > Thanks again for your help! > > Cheers, > > - Ben > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4843 bytes Desc: S/MIME Cryptographic Signature URL: From ben at well-typed.com Sat Dec 22 17:40:48 2018 From: ben at well-typed.com (Ben Gamari) Date: Sat, 22 Dec 2018 12:40:48 -0500 Subject: [GHC DevOps Group] CircleCI status In-Reply-To: References: <8736r2tjpm.fsf@smart-cactus.org> <87tvjgsvp4.fsf@smart-cactus.org> <87woo1o9b2.fsf@smart-cactus.org> Message-ID: <87r2e9o3xg.fsf@smart-cactus.org> Ccing our Windows czar, Tamar. "Greg Steuck (Sh-toy-k)" writes: > FWIW, lld seems pretty well supported on Windows > . We have some evidence that lld > is faster than gold and ld.bfd on Linux. > Indeed that is the option that I am investigating. I have opened #16084 [1] and an msys2 [2] ticket to track this. Unfortunately, GCC shipped with msys2 doesn't currently support -fuse-ld=lld, so we are stuck with ld.bfd until GCC 9 is released. However, even then it sounds like lld <8 and mingw aren't known to interact well. All-in-all, moving to lld sounds like a good long-term direction but won't be viable for at least a few months. Cheers, - Ben [1] https://ghc.haskell.org/trac/ghc/ticket/16084 [2] https://github.com/Alexpux/MSYS2-packages/issues/1531 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From lonetiger at gmail.com Sat Dec 22 18:05:15 2018 From: lonetiger at gmail.com (Phyx) Date: Sat, 22 Dec 2018 18:05:15 +0000 Subject: [GHC DevOps Group] CircleCI status In-Reply-To: <87r2e9o3xg.fsf@smart-cactus.org> References: <8736r2tjpm.fsf@smart-cactus.org> <87tvjgsvp4.fsf@smart-cactus.org> <87woo1o9b2.fsf@smart-cactus.org> <87r2e9o3xg.fsf@smart-cactus.org> Message-ID: On Sat, Dec 22, 2018 at 5:40 PM Ben Gamari wrote: > Ccing our Windows czar, Tamar. > > > "Greg Steuck (Sh-toy-k)" writes: > > > FWIW, lld seems pretty well supported on Windows > > . We have some evidence that > lld > > is faster than gold and ld.bfd on Linux. > > > Indeed that is the option that I am investigating. I have opened #16084 > [1] and an msys2 [2] ticket to track this. Unfortunately, GCC shipped > with msys2 doesn't currently support -fuse-ld=lld, so we are stuck with > ld.bfd until GCC 9 is released. However, even then it sounds like > lld <8 and mingw aren't known to interact well. > I wasn't even aware they added a mingw front-end back into LLD, this does make it a viable option. That said I also have a somewhat optimized binutils port here https://github.com/Mistuke/binutils-gdb The patch for fuse-ld=lld support is a trivial backport. I can easily build a GCC 8 release with it included and a tip of trunk LLD 8 if you'd like to try them out. I wouldn't recommend GCC 9 atm as we're in stage 3 and things are routinely broken.. > > All-in-all, moving to lld sounds like a good long-term direction but > won't be viable for at least a few months. > Yes adding it as an extra option and maybe default is good, especially as it has PDB support, making life on Windows much much easier, but we should keep bfd support as well as bfd tends to have better obscure feature support, for those that need it. Cheers, Tamar > > Cheers, > > - Ben > > > [1] https://ghc.haskell.org/trac/ghc/ticket/16084 > [2] https://github.com/Alexpux/MSYS2-packages/issues/1531 > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben at well-typed.com Sat Dec 22 19:21:09 2018 From: ben at well-typed.com (Ben Gamari) Date: Sat, 22 Dec 2018 14:21:09 -0500 Subject: [GHC DevOps Group] CircleCI status In-Reply-To: References: <8736r2tjpm.fsf@smart-cactus.org> <87tvjgsvp4.fsf@smart-cactus.org> <87woo1o9b2.fsf@smart-cactus.org> <87r2e9o3xg.fsf@smart-cactus.org> Message-ID: <87o99dnza7.fsf@smart-cactus.org> Phyx writes: > On Sat, Dec 22, 2018 at 5:40 PM Ben Gamari wrote: > >> Ccing our Windows czar, Tamar. >> >> >> "Greg Steuck (Sh-toy-k)" writes: >> >> > FWIW, lld seems pretty well supported on Windows >> > . We have some evidence that >> lld >> > is faster than gold and ld.bfd on Linux. >> > >> Indeed that is the option that I am investigating. I have opened #16084 >> [1] and an msys2 [2] ticket to track this. Unfortunately, GCC shipped >> with msys2 doesn't currently support -fuse-ld=lld, so we are stuck with >> ld.bfd until GCC 9 is released. However, even then it sounds like >> lld <8 and mingw aren't known to interact well. >> > > I wasn't even aware they added a mingw front-end back into LLD, this does > make it a viable option. > That said I also have a somewhat optimized binutils port here > https://github.com/Mistuke/binutils-gdb > > The patch for fuse-ld=lld support is a trivial backport. I can easily build > a GCC 8 release with it included and > a tip of trunk LLD 8 if you'd like to try them out. That would be fantastic. I quickly looked at what would be involved to do this myself. The process appears to be fairly straightforward However, it's a bit unclear which directory in the MSYS2-packages repository corresponds to the mingw-w64-$(uname -m)-gcc toolchain we use for builds. My guess is that [1] is the relevant package. Can you confirm? > I wouldn't recommend GCC 9 atm as we're in stage 3 and things are > routinely broken.. > Cheers, - Ben [1] https://github.com/Alexpux/MSYS2-packages/blob/master/mingw-w64-cross-gcc/PKGBUILD -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From lonetiger at gmail.com Sat Dec 22 19:36:58 2018 From: lonetiger at gmail.com (Phyx) Date: Sat, 22 Dec 2018 19:36:58 +0000 Subject: [GHC DevOps Group] CircleCI status In-Reply-To: <87o99dnza7.fsf@smart-cactus.org> References: <8736r2tjpm.fsf@smart-cactus.org> <87tvjgsvp4.fsf@smart-cactus.org> <87woo1o9b2.fsf@smart-cactus.org> <87r2e9o3xg.fsf@smart-cactus.org> <87o99dnza7.fsf@smart-cactus.org> Message-ID: On Sat, Dec 22, 2018 at 7:21 PM Ben Gamari wrote: > Phyx writes: > > > On Sat, Dec 22, 2018 at 5:40 PM Ben Gamari wrote: > > > >> Ccing our Windows czar, Tamar. > >> > >> > >> "Greg Steuck (Sh-toy-k)" writes: > >> > >> > FWIW, lld seems pretty well supported on Windows > >> > . We have some evidence > that > >> lld > >> > is faster than gold and ld.bfd on Linux. > >> > > >> Indeed that is the option that I am investigating. I have opened #16084 > >> [1] and an msys2 [2] ticket to track this. Unfortunately, GCC shipped > >> with msys2 doesn't currently support -fuse-ld=lld, so we are stuck with > >> ld.bfd until GCC 9 is released. However, even then it sounds like > >> lld <8 and mingw aren't known to interact well. > >> > > > > I wasn't even aware they added a mingw front-end back into LLD, this does > > make it a viable option. > > That said I also have a somewhat optimized binutils port here > > https://github.com/Mistuke/binutils-gdb > > > > The patch for fuse-ld=lld support is a trivial backport. I can easily > build > > a GCC 8 release with it included and > > a tip of trunk LLD 8 if you'd like to try them out. > > That would be fantastic. I quickly looked at what would be involved to > do this myself. The process appears to be fairly straightforward > However, it's a bit unclear which directory in the MSYS2-packages > repository corresponds to the mingw-w64-$(uname -m)-gcc toolchain we use > for builds. My guess is that [1] is the relevant package. Can you > confirm? > No, that one is the cross compilers for MSYS2, e.g. when you want to cross compile from the posix emulated environment to native Windows. The cross builds still rely on the msys2 runtime so won't be usable I think. We use the full native bootstrapped builds [1]. Note that this is already at GCC8, we use GCC 7 in GHC atm, so a validate build may not work due to the new warnings added, so require you to turn off -Werror for the experiment. We haven't upgraded to GCC8 as they haven't published a GCC8 for the 32 bit builds as Ada is currently broken. [1] https://github.com/Alexpux/MINGW-packages/tree/master/mingw-w64-gcc > > > I wouldn't recommend GCC 9 atm as we're in stage 3 and things are > > routinely broken.. > > > > Cheers, > > - Ben > > [1] > https://github.com/Alexpux/MSYS2-packages/blob/master/mingw-w64-cross-gcc/PKGBUILD > -------------- next part -------------- An HTML attachment was scrubbed... URL: From moritz.angermann at gmail.com Sat Dec 22 23:00:56 2018 From: moritz.angermann at gmail.com (Moritz Angermann) Date: Sun, 23 Dec 2018 07:00:56 +0800 Subject: [GHC DevOps Group] CircleCI status In-Reply-To: References: <8736r2tjpm.fsf@smart-cactus.org> <87tvjgsvp4.fsf@smart-cactus.org> <87woo1o9b2.fsf@smart-cactus.org> <87r2e9o3xg.fsf@smart-cactus.org> <87o99dnza7.fsf@smart-cactus.org> Message-ID: <9C4DF705-98A7-44E8-B399-7F99FA7FF686@gmail.com> What prevents us in principle to cross compile ghc on Linux for Windows? We’d still want to use the built artifacts and run the testsuite on Windows I think, but building ghc? Sent from my iPhone > On 23 Dec 2018, at 3:36 AM, Phyx wrote: > > > >> On Sat, Dec 22, 2018 at 7:21 PM Ben Gamari wrote: >> Phyx writes: >> >> > On Sat, Dec 22, 2018 at 5:40 PM Ben Gamari wrote: >> > >> >> Ccing our Windows czar, Tamar. >> >> >> >> >> >> "Greg Steuck (Sh-toy-k)" writes: >> >> >> >> > FWIW, lld seems pretty well supported on Windows >> >> > . We have some evidence that >> >> lld >> >> > is faster than gold and ld.bfd on Linux. >> >> > >> >> Indeed that is the option that I am investigating. I have opened #16084 >> >> [1] and an msys2 [2] ticket to track this. Unfortunately, GCC shipped >> >> with msys2 doesn't currently support -fuse-ld=lld, so we are stuck with >> >> ld.bfd until GCC 9 is released. However, even then it sounds like >> >> lld <8 and mingw aren't known to interact well. >> >> >> > >> > I wasn't even aware they added a mingw front-end back into LLD, this does >> > make it a viable option. >> > That said I also have a somewhat optimized binutils port here >> > https://github.com/Mistuke/binutils-gdb >> > >> > The patch for fuse-ld=lld support is a trivial backport. I can easily build >> > a GCC 8 release with it included and >> > a tip of trunk LLD 8 if you'd like to try them out. >> >> That would be fantastic. I quickly looked at what would be involved to >> do this myself. The process appears to be fairly straightforward >> However, it's a bit unclear which directory in the MSYS2-packages >> repository corresponds to the mingw-w64-$(uname -m)-gcc toolchain we use >> for builds. My guess is that [1] is the relevant package. Can you >> confirm? > > No, that one is the cross compilers for MSYS2, e.g. when you want to cross compile from > the posix emulated environment to native Windows. The cross builds still rely on the msys2 > runtime so won't be usable I think. > > We use the full native bootstrapped builds [1]. Note that this is already at GCC8, > we use GCC 7 in GHC atm, so a validate build may not work due to the new warnings added, > so require you to turn off -Werror for the experiment. We haven't upgraded to GCC8 as they > haven't published a GCC8 for the 32 bit builds as Ada is currently broken. > > [1] https://github.com/Alexpux/MINGW-packages/tree/master/mingw-w64-gcc > >> >> > I wouldn't recommend GCC 9 atm as we're in stage 3 and things are >> > routinely broken.. >> > >> >> Cheers, >> >> - Ben >> >> [1] https://github.com/Alexpux/MSYS2-packages/blob/master/mingw-w64-cross-gcc/PKGBUILD > _______________________________________________ > Ghc-devops-group mailing list > Ghc-devops-group at haskell.org > https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devops-group -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben at well-typed.com Sun Dec 23 00:31:27 2018 From: ben at well-typed.com (Ben Gamari) Date: Sat, 22 Dec 2018 19:31:27 -0500 Subject: [GHC DevOps Group] CircleCI status In-Reply-To: <9C4DF705-98A7-44E8-B399-7F99FA7FF686@gmail.com> References: <8736r2tjpm.fsf@smart-cactus.org> <87tvjgsvp4.fsf@smart-cactus.org> <87woo1o9b2.fsf@smart-cactus.org> <87r2e9o3xg.fsf@smart-cactus.org> <87o99dnza7.fsf@smart-cactus.org> <9C4DF705-98A7-44E8-B399-7F99FA7FF686@gmail.com> Message-ID: <87lg4hnkx1.fsf@smart-cactus.org> Moritz Angermann writes: > What prevents us in principle to cross compile ghc on Linux for > Windows? We’d still want to use the built artifacts and run the > testsuite on Windows I think, but building ghc? > We currently cannot run the full testsuite on a binary distribution due to #13897 (essentially we don't ship a few utilities which the testsuite would require; Alp is looking at fixing this in Hadrian). Moreover, we can't run Haddock when cross-compiling which means that we would need to find a way to skip the haddock performance tests. However, in general I think it is important that Tier 1 platforms be tested in a non-cross-compiled environment. Afterall, this is the environment in which we need to build binary distributions (for the above-mentioned Haddock issue, among others). Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From lonetiger at gmail.com Sun Dec 23 04:51:24 2018 From: lonetiger at gmail.com (Phyx) Date: Sun, 23 Dec 2018 04:51:24 +0000 Subject: [GHC DevOps Group] CircleCI status In-Reply-To: <87lg4hnkx1.fsf@smart-cactus.org> References: <8736r2tjpm.fsf@smart-cactus.org> <87tvjgsvp4.fsf@smart-cactus.org> <87woo1o9b2.fsf@smart-cactus.org> <87r2e9o3xg.fsf@smart-cactus.org> <87o99dnza7.fsf@smart-cactus.org> <9C4DF705-98A7-44E8-B399-7F99FA7FF686@gmail.com> <87lg4hnkx1.fsf@smart-cactus.org> Message-ID: On Sun, Dec 23, 2018, 00:31 Ben Gamari wrote: > Moritz Angermann writes: > > > What prevents us in principle to cross compile ghc on Linux for > > Windows? We’d still want to use the built artifacts and run the > > testsuite on Windows I think, but building ghc? > > > We currently cannot run the full testsuite on a binary distribution due to > #13897 (essentially we don't ship a few utilities which the testsuite would > require; Alp is looking at fixing this in Hadrian). > > Moreover, we can't run Haddock when cross-compiling which means that we > would need to find a way to skip the haddock performance tests. > > However, in general I think it is important that Tier 1 platforms be > tested in a non-cross-compiled environment. Afterall, this is the > environment in which we need to build binary distributions (for the > above-mentioned Haddock issue, among others). > And a bootstrap is representative of real code. This catches more issues than a bunch of small tests. The resulting binaries won't be representative of the code we ship to users as the cross compiler won't contain the patch (or any other optimization in master) unless you build the cross first at which point you're increasing your Linux coverage while reducing Windows. Secondly Windows is statically linked, this means you'll end up linking against library code from the cross tool chain instead of the native. The cross tool chains are often way older as they are just there to bootstrap builds. So you have a significant reduction in code coverage. > Cheers, > > - Ben > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lonetiger at gmail.com Sun Dec 23 21:17:55 2018 From: lonetiger at gmail.com (Phyx) Date: Sun, 23 Dec 2018 21:17:55 +0000 Subject: [GHC DevOps Group] CircleCI status In-Reply-To: References: <8736r2tjpm.fsf@smart-cactus.org> <87tvjgsvp4.fsf@smart-cactus.org> <87woo1o9b2.fsf@smart-cactus.org> <87r2e9o3xg.fsf@smart-cactus.org> <87o99dnza7.fsf@smart-cactus.org> Message-ID: Hi Ben, In case you haven't built one yet, here's a GCC 8 build with the patch back-ported. https://mistuke.blob.core.windows.net/binaries/releases/mingw-w64-x86_64-gcc-libs-8.2.1+20181214-1-any.pkg.tar.xz https://mistuke.blob.core.windows.net/binaries/releases/mingw-w64-x86_64-gcc-8.2.1+20181214-1-any.pkg.tar.xz I'm still trying to build lld but CMAKE is not playing along. Will let it build everything overnight. Regards, Tamar On Sat, Dec 22, 2018 at 7:36 PM Phyx wrote: > > > On Sat, Dec 22, 2018 at 7:21 PM Ben Gamari wrote: > >> Phyx writes: >> >> > On Sat, Dec 22, 2018 at 5:40 PM Ben Gamari wrote: >> > >> >> Ccing our Windows czar, Tamar. >> >> >> >> >> >> "Greg Steuck (Sh-toy-k)" writes: >> >> >> >> > FWIW, lld seems pretty well supported on Windows >> >> > . We have some evidence >> that >> >> lld >> >> > is faster than gold and ld.bfd on Linux. >> >> > >> >> Indeed that is the option that I am investigating. I have opened #16084 >> >> [1] and an msys2 [2] ticket to track this. Unfortunately, GCC shipped >> >> with msys2 doesn't currently support -fuse-ld=lld, so we are stuck with >> >> ld.bfd until GCC 9 is released. However, even then it sounds like >> >> lld <8 and mingw aren't known to interact well. >> >> >> > >> > I wasn't even aware they added a mingw front-end back into LLD, this >> does >> > make it a viable option. >> > That said I also have a somewhat optimized binutils port here >> > https://github.com/Mistuke/binutils-gdb >> > >> > The patch for fuse-ld=lld support is a trivial backport. I can easily >> build >> > a GCC 8 release with it included and >> > a tip of trunk LLD 8 if you'd like to try them out. >> >> That would be fantastic. I quickly looked at what would be involved to >> do this myself. The process appears to be fairly straightforward >> However, it's a bit unclear which directory in the MSYS2-packages >> repository corresponds to the mingw-w64-$(uname -m)-gcc toolchain we use >> for builds. My guess is that [1] is the relevant package. Can you >> confirm? >> > > No, that one is the cross compilers for MSYS2, e.g. when you want to cross > compile from > the posix emulated environment to native Windows. The cross builds still > rely on the msys2 > runtime so won't be usable I think. > > We use the full native bootstrapped builds [1]. Note that this is already > at GCC8, > we use GCC 7 in GHC atm, so a validate build may not work due to the new > warnings added, > so require you to turn off -Werror for the experiment. We haven't upgraded > to GCC8 as they > haven't published a GCC8 for the 32 bit builds as Ada is currently broken. > > [1] https://github.com/Alexpux/MINGW-packages/tree/master/mingw-w64-gcc > > >> >> > I wouldn't recommend GCC 9 atm as we're in stage 3 and things are >> > routinely broken.. >> > >> >> Cheers, >> >> - Ben >> >> [1] >> https://github.com/Alexpux/MSYS2-packages/blob/master/mingw-w64-cross-gcc/PKGBUILD >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben at well-typed.com Mon Dec 24 00:06:36 2018 From: ben at well-typed.com (Ben Gamari) Date: Sun, 23 Dec 2018 19:06:36 -0500 Subject: [GHC DevOps Group] CircleCI status In-Reply-To: References: <8736r2tjpm.fsf@smart-cactus.org> <87tvjgsvp4.fsf@smart-cactus.org> <87woo1o9b2.fsf@smart-cactus.org> <87r2e9o3xg.fsf@smart-cactus.org> <87o99dnza7.fsf@smart-cactus.org> Message-ID: <87efa7okjb.fsf@smart-cactus.org> Phyx writes: > Hi Ben, > > In case you haven't built one yet, here's a GCC 8 build with the patch > back-ported. > > https://mistuke.blob.core.windows.net/binaries/releases/mingw-w64-x86_64-gcc-libs-8.2.1+20181214-1-any.pkg.tar.xz > https://mistuke.blob.core.windows.net/binaries/releases/mingw-w64-x86_64-gcc-8.2.1+20181214-1-any.pkg.tar.xz > > I'm still trying to build lld but CMAKE is not playing along. Will let it > build everything overnight. > Thanks Tamar! Unfortunately, it sounds [1] like lld won't be a workable solution for quite a while as GHC apparently produces relocations which lld does not yet support. I'm trying to build your binutils branch currently. Cheers, - Ben [1] https://ghc.haskell.org/trac/ghc/ticket/16084#comment:5 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From ben at well-typed.com Thu Dec 27 06:27:40 2018 From: ben at well-typed.com (Ben Gamari) Date: Thu, 27 Dec 2018 01:27:40 -0500 Subject: [GHC DevOps Group] Welcome to GitLab! Message-ID: <87a7krscvf.fsf@smart-cactus.org> TL;DR. https://gitlab.haskell.org/ghc/ghc.git is now the official upstream GHC repository. Various introductory notes are discussed. Let me know if you have any trouble. Also, please do verify the correctness of the email address associated with your Trac account in the next few weeks. It will be used to map users when we transition Trac tickets to GitLab. Hello everyone, I am happy to announce that CI on GHC's GitLab instance [1] is now stable. At this point https://gitlab.haskell.org/ghc/ghc.git is to be considered the official upstream repository of GHC. The rest of this email is meant to serve as a brief introduction and status update. It can also be viewed on the GitLab Wiki [2]. [1] https://gitlab.haskell.org/ [2] https://gitlab.haskell.org/ghc/ghc/wikis/welcome # Getting started To get started on GitLab you will first want to either create a new account [1] or login with your GitHub credentials [2]. Once you have an account you should add an SSH key [3] so that you can push to your repositories. If you currently have commit rights to GHC notify me (Ben Gamari) of your user name so I can grant you similar rights in GitLab. [1] https://gitlab.haskell.org/users/sign_in [2] https://gitlab.haskell.org/users/auth/github [3] https://gitlab.haskell.org/profile/keys # Updating your development environment You can updated existing working directory (assuming the usual upstream remote name of `origin`) for the new upstream repository location by running the following: git remote set-url origin https://gitlab.haskell.org/ghc/ghc.git git remote set-url --push origin git at gitlab.haskell.org:ghc/ghc This is all that should be necessary; a quick `git pull origin master` should verify that everything is working as expected. # Continuous integration Continuous integration is now provided by GitLab's native continuous integration infrastructure. We currently test a variety of configurations, including many that neither Phabricator nor CircleCI/Appveyor previously tested (see [1] for an example run): * With the make build system: * x86_64/Linux on Fedora 27, Debian 8, and Debian 9 * i386/Linux on Debian 9 * aarch64/Linux on Debian 9 (currently broken due to a variety of issues) * x86_64/Windows * x86_64/Darwin * x86_64/Linux on Debian 9 in a few special configurations: * unregisterised (still a bit fragile due to #16085) * integer-simple * building GHC with -fllvm * With Hadrian: * x86_64/Linux on Debian 9 * x86_64/Windows (currently broken due to #15950) We also run a slightly larger set of jobs on a nightly basis. Note that binary distributions are saved from most builds and are available for download for a few weeks (we may put in place a longer retention policy for some builds in the future). There are admittedly a few kinks that we are still working out, particularly in the case of Windows (specifically the long build times seen on Windows). If you suspect you are seeing spurious build failures do let us know. To make the best use of our limited computational resources our CI builds occur in three stages: * lint: the style and correctness checkers which would previously be run by `arc lint` and `git push` * build: Debian 9 Linux x86_64 built with make and Hadrian * full-build: the remaining configurations If a build fails at an earlier phase no further phases will be run. [1] https://gitlab.haskell.org/ghc/ghc/pipelines/568 # Structuring your merge request With the transition to GitLab GHC is moving to a model similar to that used by GitHub. If you have a Differential on Phabricator we will finish review there. However, please post new patches as merge requests on GitLab. Note that Phabricator and GitLab have quite different models for handling patches. Under Phabricator a Differential is a single patch with no further structure; larger changes can be composed of multiple dependent Differentials. Under GitLab's model a merge request is a git branch consisting of one or more patches. Larger changes can be handled in one of two ways: a. a set of dependent merge requests, each of which to be squashed when merged. b. a single branch with each atomic change made in a single, buildable commit Due to the difficulty of maintaining dependent merge requests, I would recommend that contributors making larger changes use method (b). # Submitting your merge request for review Depending upon whether you have push rights to the GHC repository there are two ways to submit a merge request: * if you have push access you can push a branch directly to git at gitlab.haskell.org:ghc/ghc.git and open merge request. In this case please do follow the usual branch naming conventions: * prefix all branch names with `wip/` * if you are fixing a particular ticket consider using the name `wip/TNNNN` * if not you can create a fork using the "Fork" button on the project page [1] and push your branch there In either case after you have pushed your branch open a merge request against ghc/ghc [2]. [1] https://gitlab.haskell.org/ghc/ghc/forks/new [2] https://gitlab.haskell.org/ghc/ghc/merge_requests/new # Reviewing and merging merge requests As always, all contributors are encouraged to help review proposed changes. If you are unfamiliar with GitLab's review interface please see GitLab's user documentation [1]. Here are a few quick highlights for those who are familiar with GitHub but haven't yet used GitLab: * As with GitHub, GitLab supports both inline and out-of-line comments. * Comments that are actionable (known as "discussions") can be marked as resolved and collapsed. * Comments can be left on both changed and unchanged lines * Revisions of a merge request can be viewed and compared using the two drop-down menus at the top of the Changes tab * Merge requests can require approvals from particular users before considered as mergable * Merge requests can be placed in "merge when CI passes" state, which will cause merge requests to be merged as soon as they are green From this point onward all changes to GHC will be merged via GitLab's merge requests facility and must pass CI before being merged. To ensure that GHC's git history remains linear ghc/ghc will use GitLab's "fast-forward without a merge commit" merge strategy. Consequently you will be asked to rebase merge requests which are not fast-forward merges before merging (a convenient "Rebase" button will appear if the rebase can be carried out without conflicts. [1] https://gitlab.com/help/user/discussions/index.md#discussions # Status of the Trac migration Tobias will be continuing work on the Trac ticket migration after a bit of a holiday break. Hopefully by mid-January we will be able to move forward on this part of the migration; I will share more details about this as they develop. In the meantime, users of Trac should check and possibly update the email address associated with their account [1]. This address will be used to correlate Trac users with their GitLab equivalents so the correctness of this address will be important in preserving attribution information during the Trac import. [1] https://ghc.haskell.org/trac/ghc/prefs # Next steps We are actively working on cleaning up a few remaining issues with CI: * build times are still very long on Windows, despite the fact that we are only building the `quick` build flavour on that platform; consequently GitLab CI Windows builds do sometimes timeout when we are faced with long build queues. * we at times run low on disk space on our Windows builder runners, resulting in occasional spurious build failures * Appveyor builds (which are supposed to supplement the native GitLab builds) rarely seem to finish GitLab upstream has been incredibly supportive of our transition effort and has expressed interest in assisting us with issues that we encounter. Our current requests can be found on our migration effort's tracking ticket [1]. If you find any additional bugs or workflows that could be improved please do let me know and I can raise the matter with GitLab. [1] https://gitlab.com/gitlab-org/gitlab-ce/issues/55039 # Acknowledgments I would like to acknowledge several parties for their contributions to this effort: * Packet.net and Google X for their generous donation of hosting for continuous integration and web hosting * GitLab and their Open Source program for many productive discussions, their generous support, and the GitLab Ultimate license used by gitlab.haskell.org. * Davean Scies for his help procuring the hosting services that power our continuous integration. * Tweag.io for their offer of help and advice * Matthew Pickering, Alp Mestangullari, Tobias Dammers for their work in setting up the new instance, sorting out the details of the migration, and debugging problems when they arose Finally, thanks to GHC's contributors for their patience during this transition; it has been a long process which has stolen a significant amount of attention from other matters. My apologies we have been a bit less responsive than usual in code review and ticket triage over the past month or two. Regardless, I am hopeful that this wait will be worthwhile. # Final thoughts This is not only a milestone for the GitLab migration but also for GHC itself. For the first time GHC has fully-automated testing, proposed patch CI, and release generation across the full range of Tier 1 configurations it supports, with passing builds in all cases. We are very excited to begin this next chapter of GHC's development and are looking forward to your feedback on how we can further improve our new infrastructure. Onward and upwards! Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From alan.zimm at gmail.com Thu Dec 27 08:10:47 2018 From: alan.zimm at gmail.com (Alan & Kim Zimmerman) Date: Thu, 27 Dec 2018 10:10:47 +0200 Subject: [GHC DevOps Group] Welcome to GitLab! In-Reply-To: <87a7krscvf.fsf@smart-cactus.org> References: <87a7krscvf.fsf@smart-cactus.org> Message-ID: Congratulations, and well done. Changing basic infrastructure is a huge and thankless job. Alan On Thu, 27 Dec 2018 at 08:27, Ben Gamari wrote: > TL;DR. https://gitlab.haskell.org/ghc/ghc.git is now the official > upstream GHC repository. Various introductory notes are > discussed. Let me know if you have any trouble. > > Also, please do verify the correctness of the email address > associated with your Trac account in the next few weeks. It will > be used to map users when we transition Trac tickets to GitLab. > > > > Hello everyone, > > I am happy to announce that CI on GHC's GitLab instance [1] is now > stable. At this point https://gitlab.haskell.org/ghc/ghc.git is to be > considered the official upstream repository of GHC. > > The rest of this email is meant to serve as a brief introduction and > status update. It can also be viewed on the GitLab Wiki [2]. > > [1] https://gitlab.haskell.org/ > [2] https://gitlab.haskell.org/ghc/ghc/wikis/welcome > > > # Getting started > > To get started on GitLab you will first want to either create a new account > [1] or login with your GitHub credentials [2]. > > Once you have an account you should add an SSH key [3] so that you can push > to your repositories. If you currently have commit rights to GHC notify me > (Ben Gamari) of your user name so I can grant you similar rights in GitLab. > > > [1] https://gitlab.haskell.org/users/sign_in > [2] https://gitlab.haskell.org/users/auth/github > [3] https://gitlab.haskell.org/profile/keys > > > # Updating your development environment > > You can updated existing working directory (assuming the usual upstream > remote name of `origin`) for the new upstream repository location by > running the following: > > git remote set-url origin https://gitlab.haskell.org/ghc/ghc.git > git remote set-url --push origin git at gitlab.haskell.org:ghc/ghc > > This is all that should be necessary; a quick `git pull origin master` > should verify that everything is working as expected. > > > # Continuous integration > > Continuous integration is now provided by GitLab's native continuous > integration infrastructure. We currently test a variety of > configurations, including many that neither Phabricator nor > CircleCI/Appveyor previously tested (see [1] for an example run): > > * With the make build system: > * x86_64/Linux on Fedora 27, Debian 8, and Debian 9 > * i386/Linux on Debian 9 > * aarch64/Linux on Debian 9 (currently broken due to a variety of > issues) > * x86_64/Windows > * x86_64/Darwin > * x86_64/Linux on Debian 9 in a few special configurations: > * unregisterised (still a bit fragile due to #16085) > * integer-simple > * building GHC with -fllvm > * With Hadrian: > * x86_64/Linux on Debian 9 > * x86_64/Windows (currently broken due to #15950) > > We also run a slightly larger set of jobs on a nightly basis. Note that > binary distributions are saved from most builds and are available for > download for a few weeks (we may put in place a longer retention policy > for some builds in the future). > > There are admittedly a few kinks that we are still working out, > particularly in the case of Windows (specifically the long build times > seen on Windows). If you suspect you are seeing spurious build failures > do let us know. > > To make the best use of our limited computational resources our CI > builds occur in three stages: > > * lint: the style and correctness checkers which would previously be > run by `arc lint` and `git push` > > * build: Debian 9 Linux x86_64 built with make and Hadrian > > * full-build: the remaining configurations > > If a build fails at an earlier phase no further phases will be run. > > > [1] https://gitlab.haskell.org/ghc/ghc/pipelines/568 > > > # Structuring your merge request > > With the transition to GitLab GHC is moving to a model similar to that > used by GitHub. If you have a Differential on Phabricator we will finish > review there. However, please post new patches as merge requests on > GitLab. > > Note that Phabricator and GitLab have quite different models for > handling patches. Under Phabricator a Differential is a single patch > with no further structure; larger changes can be composed of multiple > dependent Differentials. > > Under GitLab's model a merge request is a git branch consisting of > one or more patches. Larger changes can be handled in one of two ways: > > a. a set of dependent merge requests, each of which to be squashed when > merged. > > b. a single branch with each atomic change made in a single, buildable > commit > > Due to the difficulty of maintaining dependent merge requests, I would > recommend that contributors making larger changes use method (b). > > > # Submitting your merge request for review > > Depending upon whether you have push rights to the GHC repository there > are two ways to submit a merge request: > > * if you have push access you can push a branch directly to > git at gitlab.haskell.org:ghc/ghc.git and open merge request. > > In this case please do follow the usual branch naming conventions: > > * prefix all branch names with `wip/` > > * if you are fixing a particular ticket consider using the name > `wip/TNNNN` > > * if not you can create a fork using the "Fork" button on the project > page [1] and push your branch there > > In either case after you have pushed your branch open a merge request > against ghc/ghc [2]. > > [1] https://gitlab.haskell.org/ghc/ghc/forks/new > [2] https://gitlab.haskell.org/ghc/ghc/merge_requests/new > > > # Reviewing and merging merge requests > > As always, all contributors are encouraged to help review proposed > changes. If you are unfamiliar with GitLab's review interface please see > GitLab's user documentation [1]. Here are a few quick highlights for > those who are familiar with GitHub but haven't yet used GitLab: > > * As with GitHub, GitLab supports both inline and out-of-line comments. > > * Comments that are actionable (known as "discussions") can be marked > as resolved and collapsed. > > * Comments can be left on both changed and unchanged lines > > * Revisions of a merge request can be viewed and compared using the > two drop-down menus at the top of the Changes tab > > * Merge requests can require approvals from particular users before > considered as mergable > > * Merge requests can be placed in "merge when CI passes" state, which > will cause merge requests to be merged as soon as they are green > > From this point onward all changes to GHC will be merged via > GitLab's merge requests facility and must pass CI before being merged. > To ensure that GHC's git history remains linear ghc/ghc will use GitLab's > "fast-forward without a merge commit" merge strategy. Consequently you > will be asked to rebase merge requests which are not fast-forward merges > before merging (a convenient "Rebase" button will appear if the rebase > can be carried out without conflicts. > > [1] https://gitlab.com/help/user/discussions/index.md#discussions > > > # Status of the Trac migration > > Tobias will be continuing work on the Trac ticket migration after a bit > of a holiday break. Hopefully by mid-January we will be able to move > forward on this part of the migration; I will share more details about > this as they develop. > > In the meantime, users of Trac should check and possibly update the > email address associated with their account [1]. This address will be > used to correlate Trac users with their GitLab equivalents so the > correctness of this address will be important in preserving attribution > information during the Trac import. > > [1] https://ghc.haskell.org/trac/ghc/prefs > > > # Next steps > > We are actively working on cleaning up a few remaining issues with CI: > > * build times are still very long on Windows, despite the fact that we > are only building the `quick` build flavour on that platform; > consequently GitLab CI Windows builds do sometimes timeout > when we are faced with long build queues. > > * we at times run low on disk space on our Windows builder runners, > resulting in occasional spurious build failures > > * Appveyor builds (which are supposed to supplement the native GitLab > builds) rarely seem to finish > > GitLab upstream has been incredibly supportive of our transition effort > and has expressed interest in assisting us with issues that we > encounter. Our current requests can be found on our migration effort's > tracking ticket [1]. If you find any additional bugs or workflows that > could be improved please do let me know and I can raise the matter with > GitLab. > > [1] https://gitlab.com/gitlab-org/gitlab-ce/issues/55039 > > > # Acknowledgments > > I would like to acknowledge several parties for their contributions to > this effort: > > * Packet.net and Google X for their generous donation of hosting for > continuous integration and web hosting > > * GitLab and their Open Source program for many productive discussions, > their generous support, and the GitLab Ultimate license used by > gitlab.haskell.org. > > * Davean Scies for his help procuring the hosting services that power > our continuous integration. > > * Tweag.io for their offer of help and advice > > * Matthew Pickering, Alp Mestangullari, Tobias Dammers for their work > in setting up the new instance, sorting out the details of the > migration, and debugging problems when they arose > > Finally, thanks to GHC's contributors for their patience during this > transition; it has been a long process which has stolen a significant > amount of attention from other matters. My apologies we have been a bit > less responsive than usual in code review and ticket triage over the > past month or two. Regardless, I am hopeful that this wait will be > worthwhile. > > > # Final thoughts > > This is not only a milestone for the GitLab migration but also for GHC > itself. For the first time GHC has fully-automated testing, proposed > patch CI, and release generation across the full range of Tier 1 > configurations it supports, with passing builds in all cases. > > We are very excited to begin this next chapter of GHC's development and > are looking forward to your feedback on how we can further improve our > new infrastructure. Onward and upwards! > > Cheers, > > - Ben > _______________________________________________ > Ghc-devops-group mailing list > Ghc-devops-group at haskell.org > https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devops-group > -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthewtpickering at gmail.com Thu Dec 27 11:44:30 2018 From: matthewtpickering at gmail.com (Matthew Pickering) Date: Thu, 27 Dec 2018 11:44:30 +0000 Subject: [GHC DevOps Group] Welcome to GitLab! In-Reply-To: <87a7krscvf.fsf@smart-cactus.org> References: <87a7krscvf.fsf@smart-cactus.org> Message-ID: I have a patch on phabricator which has finished review and is ready to merge. https://phabricator.haskell.org/D5458 Should I put it on Gitlab as now all patches should pass CI before being merged? Matt On Thu, Dec 27, 2018 at 6:28 AM Ben Gamari wrote: > > TL;DR. https://gitlab.haskell.org/ghc/ghc.git is now the official > upstream GHC repository. Various introductory notes are > discussed. Let me know if you have any trouble. > > Also, please do verify the correctness of the email address > associated with your Trac account in the next few weeks. It will > be used to map users when we transition Trac tickets to GitLab. > > > > Hello everyone, > > I am happy to announce that CI on GHC's GitLab instance [1] is now > stable. At this point https://gitlab.haskell.org/ghc/ghc.git is to be > considered the official upstream repository of GHC. > > The rest of this email is meant to serve as a brief introduction and > status update. It can also be viewed on the GitLab Wiki [2]. > > [1] https://gitlab.haskell.org/ > [2] https://gitlab.haskell.org/ghc/ghc/wikis/welcome > > > # Getting started > > To get started on GitLab you will first want to either create a new account > [1] or login with your GitHub credentials [2]. > > Once you have an account you should add an SSH key [3] so that you can push > to your repositories. If you currently have commit rights to GHC notify me > (Ben Gamari) of your user name so I can grant you similar rights in GitLab. > > > [1] https://gitlab.haskell.org/users/sign_in > [2] https://gitlab.haskell.org/users/auth/github > [3] https://gitlab.haskell.org/profile/keys > > > # Updating your development environment > > You can updated existing working directory (assuming the usual upstream > remote name of `origin`) for the new upstream repository location by > running the following: > > git remote set-url origin https://gitlab.haskell.org/ghc/ghc.git > git remote set-url --push origin git at gitlab.haskell.org:ghc/ghc > > This is all that should be necessary; a quick `git pull origin master` > should verify that everything is working as expected. > > > # Continuous integration > > Continuous integration is now provided by GitLab's native continuous > integration infrastructure. We currently test a variety of > configurations, including many that neither Phabricator nor > CircleCI/Appveyor previously tested (see [1] for an example run): > > * With the make build system: > * x86_64/Linux on Fedora 27, Debian 8, and Debian 9 > * i386/Linux on Debian 9 > * aarch64/Linux on Debian 9 (currently broken due to a variety of > issues) > * x86_64/Windows > * x86_64/Darwin > * x86_64/Linux on Debian 9 in a few special configurations: > * unregisterised (still a bit fragile due to #16085) > * integer-simple > * building GHC with -fllvm > * With Hadrian: > * x86_64/Linux on Debian 9 > * x86_64/Windows (currently broken due to #15950) > > We also run a slightly larger set of jobs on a nightly basis. Note that > binary distributions are saved from most builds and are available for > download for a few weeks (we may put in place a longer retention policy > for some builds in the future). > > There are admittedly a few kinks that we are still working out, > particularly in the case of Windows (specifically the long build times > seen on Windows). If you suspect you are seeing spurious build failures > do let us know. > > To make the best use of our limited computational resources our CI > builds occur in three stages: > > * lint: the style and correctness checkers which would previously be > run by `arc lint` and `git push` > > * build: Debian 9 Linux x86_64 built with make and Hadrian > > * full-build: the remaining configurations > > If a build fails at an earlier phase no further phases will be run. > > > [1] https://gitlab.haskell.org/ghc/ghc/pipelines/568 > > > # Structuring your merge request > > With the transition to GitLab GHC is moving to a model similar to that > used by GitHub. If you have a Differential on Phabricator we will finish > review there. However, please post new patches as merge requests on > GitLab. > > Note that Phabricator and GitLab have quite different models for > handling patches. Under Phabricator a Differential is a single patch > with no further structure; larger changes can be composed of multiple > dependent Differentials. > > Under GitLab's model a merge request is a git branch consisting of > one or more patches. Larger changes can be handled in one of two ways: > > a. a set of dependent merge requests, each of which to be squashed when > merged. > > b. a single branch with each atomic change made in a single, buildable > commit > > Due to the difficulty of maintaining dependent merge requests, I would > recommend that contributors making larger changes use method (b). > > > # Submitting your merge request for review > > Depending upon whether you have push rights to the GHC repository there > are two ways to submit a merge request: > > * if you have push access you can push a branch directly to > git at gitlab.haskell.org:ghc/ghc.git and open merge request. > > In this case please do follow the usual branch naming conventions: > > * prefix all branch names with `wip/` > > * if you are fixing a particular ticket consider using the name > `wip/TNNNN` > > * if not you can create a fork using the "Fork" button on the project > page [1] and push your branch there > > In either case after you have pushed your branch open a merge request > against ghc/ghc [2]. > > [1] https://gitlab.haskell.org/ghc/ghc/forks/new > [2] https://gitlab.haskell.org/ghc/ghc/merge_requests/new > > > # Reviewing and merging merge requests > > As always, all contributors are encouraged to help review proposed > changes. If you are unfamiliar with GitLab's review interface please see > GitLab's user documentation [1]. Here are a few quick highlights for > those who are familiar with GitHub but haven't yet used GitLab: > > * As with GitHub, GitLab supports both inline and out-of-line comments. > > * Comments that are actionable (known as "discussions") can be marked > as resolved and collapsed. > > * Comments can be left on both changed and unchanged lines > > * Revisions of a merge request can be viewed and compared using the > two drop-down menus at the top of the Changes tab > > * Merge requests can require approvals from particular users before > considered as mergable > > * Merge requests can be placed in "merge when CI passes" state, which > will cause merge requests to be merged as soon as they are green > > From this point onward all changes to GHC will be merged via > GitLab's merge requests facility and must pass CI before being merged. > To ensure that GHC's git history remains linear ghc/ghc will use GitLab's > "fast-forward without a merge commit" merge strategy. Consequently you > will be asked to rebase merge requests which are not fast-forward merges > before merging (a convenient "Rebase" button will appear if the rebase > can be carried out without conflicts. > > [1] https://gitlab.com/help/user/discussions/index.md#discussions > > > # Status of the Trac migration > > Tobias will be continuing work on the Trac ticket migration after a bit > of a holiday break. Hopefully by mid-January we will be able to move > forward on this part of the migration; I will share more details about > this as they develop. > > In the meantime, users of Trac should check and possibly update the > email address associated with their account [1]. This address will be > used to correlate Trac users with their GitLab equivalents so the > correctness of this address will be important in preserving attribution > information during the Trac import. > > [1] https://ghc.haskell.org/trac/ghc/prefs > > > # Next steps > > We are actively working on cleaning up a few remaining issues with CI: > > * build times are still very long on Windows, despite the fact that we > are only building the `quick` build flavour on that platform; > consequently GitLab CI Windows builds do sometimes timeout > when we are faced with long build queues. > > * we at times run low on disk space on our Windows builder runners, > resulting in occasional spurious build failures > > * Appveyor builds (which are supposed to supplement the native GitLab > builds) rarely seem to finish > > GitLab upstream has been incredibly supportive of our transition effort > and has expressed interest in assisting us with issues that we > encounter. Our current requests can be found on our migration effort's > tracking ticket [1]. If you find any additional bugs or workflows that > could be improved please do let me know and I can raise the matter with > GitLab. > > [1] https://gitlab.com/gitlab-org/gitlab-ce/issues/55039 > > > # Acknowledgments > > I would like to acknowledge several parties for their contributions to > this effort: > > * Packet.net and Google X for their generous donation of hosting for > continuous integration and web hosting > > * GitLab and their Open Source program for many productive discussions, > their generous support, and the GitLab Ultimate license used by > gitlab.haskell.org. > > * Davean Scies for his help procuring the hosting services that power > our continuous integration. > > * Tweag.io for their offer of help and advice > > * Matthew Pickering, Alp Mestangullari, Tobias Dammers for their work > in setting up the new instance, sorting out the details of the > migration, and debugging problems when they arose > > Finally, thanks to GHC's contributors for their patience during this > transition; it has been a long process which has stolen a significant > amount of attention from other matters. My apologies we have been a bit > less responsive than usual in code review and ticket triage over the > past month or two. Regardless, I am hopeful that this wait will be > worthwhile. > > > # Final thoughts > > This is not only a milestone for the GitLab migration but also for GHC > itself. For the first time GHC has fully-automated testing, proposed > patch CI, and release generation across the full range of Tier 1 > configurations it supports, with passing builds in all cases. > > We are very excited to begin this next chapter of GHC's development and > are looking forward to your feedback on how we can further improve our > new infrastructure. Onward and upwards! > > Cheers, > > - Ben > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs From matthewtpickering at gmail.com Thu Dec 27 11:56:10 2018 From: matthewtpickering at gmail.com (Matthew Pickering) Date: Thu, 27 Dec 2018 11:56:10 +0000 Subject: [GHC DevOps Group] Welcome to GitLab! In-Reply-To: <87a7krscvf.fsf@smart-cactus.org> References: <87a7krscvf.fsf@smart-cactus.org> Message-ID: > To ensure that GHC's git history remains linear ghc/ghc will use GitLab's > "fast-forward without a merge commit" merge strategy. Are merge requests squashed before they are merged? It seems that the answer by default is no.. https://gitlab.com/gitlab-org/gitlab-ce/issues/27956 and the reason being that upsteam prefers "Convention over Configuration".. https://about.gitlab.com/handbook/product/#convention-over-configuration However it seems that there is a per-mr option which can be checked if you are diligent to do it for each MR. Some comments indicate that it's possible to implement a webhook to change this behaviour. Matt On Thu, Dec 27, 2018 at 6:28 AM Ben Gamari wrote: > > TL;DR. https://gitlab.haskell.org/ghc/ghc.git is now the official > upstream GHC repository. Various introductory notes are > discussed. Let me know if you have any trouble. > > Also, please do verify the correctness of the email address > associated with your Trac account in the next few weeks. It will > be used to map users when we transition Trac tickets to GitLab. > > > > Hello everyone, > > I am happy to announce that CI on GHC's GitLab instance [1] is now > stable. At this point https://gitlab.haskell.org/ghc/ghc.git is to be > considered the official upstream repository of GHC. > > The rest of this email is meant to serve as a brief introduction and > status update. It can also be viewed on the GitLab Wiki [2]. > > [1] https://gitlab.haskell.org/ > [2] https://gitlab.haskell.org/ghc/ghc/wikis/welcome > > > # Getting started > > To get started on GitLab you will first want to either create a new account > [1] or login with your GitHub credentials [2]. > > Once you have an account you should add an SSH key [3] so that you can push > to your repositories. If you currently have commit rights to GHC notify me > (Ben Gamari) of your user name so I can grant you similar rights in GitLab. > > > [1] https://gitlab.haskell.org/users/sign_in > [2] https://gitlab.haskell.org/users/auth/github > [3] https://gitlab.haskell.org/profile/keys > > > # Updating your development environment > > You can updated existing working directory (assuming the usual upstream > remote name of `origin`) for the new upstream repository location by > running the following: > > git remote set-url origin https://gitlab.haskell.org/ghc/ghc.git > git remote set-url --push origin git at gitlab.haskell.org:ghc/ghc > > This is all that should be necessary; a quick `git pull origin master` > should verify that everything is working as expected. > > > # Continuous integration > > Continuous integration is now provided by GitLab's native continuous > integration infrastructure. We currently test a variety of > configurations, including many that neither Phabricator nor > CircleCI/Appveyor previously tested (see [1] for an example run): > > * With the make build system: > * x86_64/Linux on Fedora 27, Debian 8, and Debian 9 > * i386/Linux on Debian 9 > * aarch64/Linux on Debian 9 (currently broken due to a variety of > issues) > * x86_64/Windows > * x86_64/Darwin > * x86_64/Linux on Debian 9 in a few special configurations: > * unregisterised (still a bit fragile due to #16085) > * integer-simple > * building GHC with -fllvm > * With Hadrian: > * x86_64/Linux on Debian 9 > * x86_64/Windows (currently broken due to #15950) > > We also run a slightly larger set of jobs on a nightly basis. Note that > binary distributions are saved from most builds and are available for > download for a few weeks (we may put in place a longer retention policy > for some builds in the future). > > There are admittedly a few kinks that we are still working out, > particularly in the case of Windows (specifically the long build times > seen on Windows). If you suspect you are seeing spurious build failures > do let us know. > > To make the best use of our limited computational resources our CI > builds occur in three stages: > > * lint: the style and correctness checkers which would previously be > run by `arc lint` and `git push` > > * build: Debian 9 Linux x86_64 built with make and Hadrian > > * full-build: the remaining configurations > > If a build fails at an earlier phase no further phases will be run. > > > [1] https://gitlab.haskell.org/ghc/ghc/pipelines/568 > > > # Structuring your merge request > > With the transition to GitLab GHC is moving to a model similar to that > used by GitHub. If you have a Differential on Phabricator we will finish > review there. However, please post new patches as merge requests on > GitLab. > > Note that Phabricator and GitLab have quite different models for > handling patches. Under Phabricator a Differential is a single patch > with no further structure; larger changes can be composed of multiple > dependent Differentials. > > Under GitLab's model a merge request is a git branch consisting of > one or more patches. Larger changes can be handled in one of two ways: > > a. a set of dependent merge requests, each of which to be squashed when > merged. > > b. a single branch with each atomic change made in a single, buildable > commit > > Due to the difficulty of maintaining dependent merge requests, I would > recommend that contributors making larger changes use method (b). > > > # Submitting your merge request for review > > Depending upon whether you have push rights to the GHC repository there > are two ways to submit a merge request: > > * if you have push access you can push a branch directly to > git at gitlab.haskell.org:ghc/ghc.git and open merge request. > > In this case please do follow the usual branch naming conventions: > > * prefix all branch names with `wip/` > > * if you are fixing a particular ticket consider using the name > `wip/TNNNN` > > * if not you can create a fork using the "Fork" button on the project > page [1] and push your branch there > > In either case after you have pushed your branch open a merge request > against ghc/ghc [2]. > > [1] https://gitlab.haskell.org/ghc/ghc/forks/new > [2] https://gitlab.haskell.org/ghc/ghc/merge_requests/new > > > # Reviewing and merging merge requests > > As always, all contributors are encouraged to help review proposed > changes. If you are unfamiliar with GitLab's review interface please see > GitLab's user documentation [1]. Here are a few quick highlights for > those who are familiar with GitHub but haven't yet used GitLab: > > * As with GitHub, GitLab supports both inline and out-of-line comments. > > * Comments that are actionable (known as "discussions") can be marked > as resolved and collapsed. > > * Comments can be left on both changed and unchanged lines > > * Revisions of a merge request can be viewed and compared using the > two drop-down menus at the top of the Changes tab > > * Merge requests can require approvals from particular users before > considered as mergable > > * Merge requests can be placed in "merge when CI passes" state, which > will cause merge requests to be merged as soon as they are green > > From this point onward all changes to GHC will be merged via > GitLab's merge requests facility and must pass CI before being merged. > To ensure that GHC's git history remains linear ghc/ghc will use GitLab's > "fast-forward without a merge commit" merge strategy. Consequently you > will be asked to rebase merge requests which are not fast-forward merges > before merging (a convenient "Rebase" button will appear if the rebase > can be carried out without conflicts. > > [1] https://gitlab.com/help/user/discussions/index.md#discussions > > > # Status of the Trac migration > > Tobias will be continuing work on the Trac ticket migration after a bit > of a holiday break. Hopefully by mid-January we will be able to move > forward on this part of the migration; I will share more details about > this as they develop. > > In the meantime, users of Trac should check and possibly update the > email address associated with their account [1]. This address will be > used to correlate Trac users with their GitLab equivalents so the > correctness of this address will be important in preserving attribution > information during the Trac import. > > [1] https://ghc.haskell.org/trac/ghc/prefs > > > # Next steps > > We are actively working on cleaning up a few remaining issues with CI: > > * build times are still very long on Windows, despite the fact that we > are only building the `quick` build flavour on that platform; > consequently GitLab CI Windows builds do sometimes timeout > when we are faced with long build queues. > > * we at times run low on disk space on our Windows builder runners, > resulting in occasional spurious build failures > > * Appveyor builds (which are supposed to supplement the native GitLab > builds) rarely seem to finish > > GitLab upstream has been incredibly supportive of our transition effort > and has expressed interest in assisting us with issues that we > encounter. Our current requests can be found on our migration effort's > tracking ticket [1]. If you find any additional bugs or workflows that > could be improved please do let me know and I can raise the matter with > GitLab. > > [1] https://gitlab.com/gitlab-org/gitlab-ce/issues/55039 > > > # Acknowledgments > > I would like to acknowledge several parties for their contributions to > this effort: > > * Packet.net and Google X for their generous donation of hosting for > continuous integration and web hosting > > * GitLab and their Open Source program for many productive discussions, > their generous support, and the GitLab Ultimate license used by > gitlab.haskell.org. > > * Davean Scies for his help procuring the hosting services that power > our continuous integration. > > * Tweag.io for their offer of help and advice > > * Matthew Pickering, Alp Mestangullari, Tobias Dammers for their work > in setting up the new instance, sorting out the details of the > migration, and debugging problems when they arose > > Finally, thanks to GHC's contributors for their patience during this > transition; it has been a long process which has stolen a significant > amount of attention from other matters. My apologies we have been a bit > less responsive than usual in code review and ticket triage over the > past month or two. Regardless, I am hopeful that this wait will be > worthwhile. > > > # Final thoughts > > This is not only a milestone for the GitLab migration but also for GHC > itself. For the first time GHC has fully-automated testing, proposed > patch CI, and release generation across the full range of Tier 1 > configurations it supports, with passing builds in all cases. > > We are very excited to begin this next chapter of GHC's development and > are looking forward to your feedback on how we can further improve our > new infrastructure. Onward and upwards! > > Cheers, > > - Ben > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs From me at ara.io Thu Dec 27 12:04:10 2018 From: me at ara.io (Ara Adkins) Date: Thu, 27 Dec 2018 12:04:10 +0000 Subject: [GHC DevOps Group] Welcome to GitLab! In-Reply-To: References: <87a7krscvf.fsf@smart-cactus.org> Message-ID: Congrats to Ben and everybody involved! This has been a long time coming and I’m super excited to see what it means for GHC in the future! _ara On 27 Dec 2018, at 11:56, Matthew Pickering wrote: >> To ensure that GHC's git history remains linear ghc/ghc will use GitLab's >> "fast-forward without a merge commit" merge strategy. > > Are merge requests squashed before they are merged? > > It seems that the answer by default is no.. > https://gitlab.com/gitlab-org/gitlab-ce/issues/27956 > > and the reason being that upsteam prefers "Convention over > Configuration".. > https://about.gitlab.com/handbook/product/#convention-over-configuration > > However it seems that there is a per-mr option which can be checked if > you are diligent to do it for each MR. Some comments indicate that > it's possible to implement a webhook to change this behaviour. > > Matt > >> On Thu, Dec 27, 2018 at 6:28 AM Ben Gamari wrote: >> >> TL;DR. https://gitlab.haskell.org/ghc/ghc.git is now the official >> upstream GHC repository. Various introductory notes are >> discussed. Let me know if you have any trouble. >> >> Also, please do verify the correctness of the email address >> associated with your Trac account in the next few weeks. It will >> be used to map users when we transition Trac tickets to GitLab. >> >> >> >> Hello everyone, >> >> I am happy to announce that CI on GHC's GitLab instance [1] is now >> stable. At this point https://gitlab.haskell.org/ghc/ghc.git is to be >> considered the official upstream repository of GHC. >> >> The rest of this email is meant to serve as a brief introduction and >> status update. It can also be viewed on the GitLab Wiki [2]. >> >> [1] https://gitlab.haskell.org/ >> [2] https://gitlab.haskell.org/ghc/ghc/wikis/welcome >> >> >> # Getting started >> >> To get started on GitLab you will first want to either create a new account >> [1] or login with your GitHub credentials [2]. >> >> Once you have an account you should add an SSH key [3] so that you can push >> to your repositories. If you currently have commit rights to GHC notify me >> (Ben Gamari) of your user name so I can grant you similar rights in GitLab. >> >> >> [1] https://gitlab.haskell.org/users/sign_in >> [2] https://gitlab.haskell.org/users/auth/github >> [3] https://gitlab.haskell.org/profile/keys >> >> >> # Updating your development environment >> >> You can updated existing working directory (assuming the usual upstream >> remote name of `origin`) for the new upstream repository location by >> running the following: >> >> git remote set-url origin https://gitlab.haskell.org/ghc/ghc.git >> git remote set-url --push origin git at gitlab.haskell.org:ghc/ghc >> >> This is all that should be necessary; a quick `git pull origin master` >> should verify that everything is working as expected. >> >> >> # Continuous integration >> >> Continuous integration is now provided by GitLab's native continuous >> integration infrastructure. We currently test a variety of >> configurations, including many that neither Phabricator nor >> CircleCI/Appveyor previously tested (see [1] for an example run): >> >> * With the make build system: >> * x86_64/Linux on Fedora 27, Debian 8, and Debian 9 >> * i386/Linux on Debian 9 >> * aarch64/Linux on Debian 9 (currently broken due to a variety of >> issues) >> * x86_64/Windows >> * x86_64/Darwin >> * x86_64/Linux on Debian 9 in a few special configurations: >> * unregisterised (still a bit fragile due to #16085) >> * integer-simple >> * building GHC with -fllvm >> * With Hadrian: >> * x86_64/Linux on Debian 9 >> * x86_64/Windows (currently broken due to #15950) >> >> We also run a slightly larger set of jobs on a nightly basis. Note that >> binary distributions are saved from most builds and are available for >> download for a few weeks (we may put in place a longer retention policy >> for some builds in the future). >> >> There are admittedly a few kinks that we are still working out, >> particularly in the case of Windows (specifically the long build times >> seen on Windows). If you suspect you are seeing spurious build failures >> do let us know. >> >> To make the best use of our limited computational resources our CI >> builds occur in three stages: >> >> * lint: the style and correctness checkers which would previously be >> run by `arc lint` and `git push` >> >> * build: Debian 9 Linux x86_64 built with make and Hadrian >> >> * full-build: the remaining configurations >> >> If a build fails at an earlier phase no further phases will be run. >> >> >> [1] https://gitlab.haskell.org/ghc/ghc/pipelines/568 >> >> >> # Structuring your merge request >> >> With the transition to GitLab GHC is moving to a model similar to that >> used by GitHub. If you have a Differential on Phabricator we will finish >> review there. However, please post new patches as merge requests on >> GitLab. >> >> Note that Phabricator and GitLab have quite different models for >> handling patches. Under Phabricator a Differential is a single patch >> with no further structure; larger changes can be composed of multiple >> dependent Differentials. >> >> Under GitLab's model a merge request is a git branch consisting of >> one or more patches. Larger changes can be handled in one of two ways: >> >> a. a set of dependent merge requests, each of which to be squashed when >> merged. >> >> b. a single branch with each atomic change made in a single, buildable >> commit >> >> Due to the difficulty of maintaining dependent merge requests, I would >> recommend that contributors making larger changes use method (b). >> >> >> # Submitting your merge request for review >> >> Depending upon whether you have push rights to the GHC repository there >> are two ways to submit a merge request: >> >> * if you have push access you can push a branch directly to >> git at gitlab.haskell.org:ghc/ghc.git and open merge request. >> >> In this case please do follow the usual branch naming conventions: >> >> * prefix all branch names with `wip/` >> >> * if you are fixing a particular ticket consider using the name >> `wip/TNNNN` >> >> * if not you can create a fork using the "Fork" button on the project >> page [1] and push your branch there >> >> In either case after you have pushed your branch open a merge request >> against ghc/ghc [2]. >> >> [1] https://gitlab.haskell.org/ghc/ghc/forks/new >> [2] https://gitlab.haskell.org/ghc/ghc/merge_requests/new >> >> >> # Reviewing and merging merge requests >> >> As always, all contributors are encouraged to help review proposed >> changes. If you are unfamiliar with GitLab's review interface please see >> GitLab's user documentation [1]. Here are a few quick highlights for >> those who are familiar with GitHub but haven't yet used GitLab: >> >> * As with GitHub, GitLab supports both inline and out-of-line comments. >> >> * Comments that are actionable (known as "discussions") can be marked >> as resolved and collapsed. >> >> * Comments can be left on both changed and unchanged lines >> >> * Revisions of a merge request can be viewed and compared using the >> two drop-down menus at the top of the Changes tab >> >> * Merge requests can require approvals from particular users before >> considered as mergable >> >> * Merge requests can be placed in "merge when CI passes" state, which >> will cause merge requests to be merged as soon as they are green >> >> From this point onward all changes to GHC will be merged via >> GitLab's merge requests facility and must pass CI before being merged. >> To ensure that GHC's git history remains linear ghc/ghc will use GitLab's >> "fast-forward without a merge commit" merge strategy. Consequently you >> will be asked to rebase merge requests which are not fast-forward merges >> before merging (a convenient "Rebase" button will appear if the rebase >> can be carried out without conflicts. >> >> [1] https://gitlab.com/help/user/discussions/index.md#discussions >> >> >> # Status of the Trac migration >> >> Tobias will be continuing work on the Trac ticket migration after a bit >> of a holiday break. Hopefully by mid-January we will be able to move >> forward on this part of the migration; I will share more details about >> this as they develop. >> >> In the meantime, users of Trac should check and possibly update the >> email address associated with their account [1]. This address will be >> used to correlate Trac users with their GitLab equivalents so the >> correctness of this address will be important in preserving attribution >> information during the Trac import. >> >> [1] https://ghc.haskell.org/trac/ghc/prefs >> >> >> # Next steps >> >> We are actively working on cleaning up a few remaining issues with CI: >> >> * build times are still very long on Windows, despite the fact that we >> are only building the `quick` build flavour on that platform; >> consequently GitLab CI Windows builds do sometimes timeout >> when we are faced with long build queues. >> >> * we at times run low on disk space on our Windows builder runners, >> resulting in occasional spurious build failures >> >> * Appveyor builds (which are supposed to supplement the native GitLab >> builds) rarely seem to finish >> >> GitLab upstream has been incredibly supportive of our transition effort >> and has expressed interest in assisting us with issues that we >> encounter. Our current requests can be found on our migration effort's >> tracking ticket [1]. If you find any additional bugs or workflows that >> could be improved please do let me know and I can raise the matter with >> GitLab. >> >> [1] https://gitlab.com/gitlab-org/gitlab-ce/issues/55039 >> >> >> # Acknowledgments >> >> I would like to acknowledge several parties for their contributions to >> this effort: >> >> * Packet.net and Google X for their generous donation of hosting for >> continuous integration and web hosting >> >> * GitLab and their Open Source program for many productive discussions, >> their generous support, and the GitLab Ultimate license used by >> gitlab.haskell.org. >> >> * Davean Scies for his help procuring the hosting services that power >> our continuous integration. >> >> * Tweag.io for their offer of help and advice >> >> * Matthew Pickering, Alp Mestangullari, Tobias Dammers for their work >> in setting up the new instance, sorting out the details of the >> migration, and debugging problems when they arose >> >> Finally, thanks to GHC's contributors for their patience during this >> transition; it has been a long process which has stolen a significant >> amount of attention from other matters. My apologies we have been a bit >> less responsive than usual in code review and ticket triage over the >> past month or two. Regardless, I am hopeful that this wait will be >> worthwhile. >> >> >> # Final thoughts >> >> This is not only a milestone for the GitLab migration but also for GHC >> itself. For the first time GHC has fully-automated testing, proposed >> patch CI, and release generation across the full range of Tier 1 >> configurations it supports, with passing builds in all cases. >> >> We are very excited to begin this next chapter of GHC's development and >> are looking forward to your feedback on how we can further improve our >> new infrastructure. Onward and upwards! >> >> Cheers, >> >> - Ben >> _______________________________________________ >> ghc-devs mailing list >> ghc-devs at haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs From ben at well-typed.com Thu Dec 27 15:11:09 2018 From: ben at well-typed.com (Ben Gamari) Date: Thu, 27 Dec 2018 10:11:09 -0500 Subject: [GHC DevOps Group] Welcome to GitLab! In-Reply-To: References: <87a7krscvf.fsf@smart-cactus.org> Message-ID: <871s63roms.fsf@smart-cactus.org> Matthew Pickering writes: >> To ensure that GHC's git history remains linear ghc/ghc will use GitLab's >> "fast-forward without a merge commit" merge strategy. > > Are merge requests squashed before they are merged? > > It seems that the answer by default is no.. > https://gitlab.com/gitlab-org/gitlab-ce/issues/27956 > Indeed there are not. Moreover, in the workflow that I suggest in the email not squashing is the desired behavior since each commit is atomic. > and the reason being that upsteam prefers "Convention over > Configuration".. > https://about.gitlab.com/handbook/product/#convention-over-configuration > > However it seems that there is a per-mr option which can be checked if > you are diligent to do it for each MR. Some comments indicate that > it's possible to implement a webhook to change this behaviour. > I'm not sure there is a reasonable default here; not matter what you choose it will be wrong a good fraction of the time. The current plan is to just ensure that the person who merges an MR considers whether the history it introduces is useful and choose the correct option. I believe this should work fine although I'm happy to reconsider if not. Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From matthewtpickering at gmail.com Thu Dec 27 15:29:46 2018 From: matthewtpickering at gmail.com (Matthew Pickering) Date: Thu, 27 Dec 2018 15:29:46 +0000 Subject: [GHC DevOps Group] Welcome to GitLab! In-Reply-To: <871s63roms.fsf@smart-cactus.org> References: <87a7krscvf.fsf@smart-cactus.org> <871s63roms.fsf@smart-cactus.org> Message-ID: Does this mean that the commit message doesn't come from the MR description? If I want to edit the commit message for a description I need to 1. squash the changes locally 2. force push the branch 3. Wait for CI to finish building (even though I just changed the commit message). Matt On Thu, Dec 27, 2018 at 3:11 PM Ben Gamari wrote: > > Matthew Pickering writes: > > >> To ensure that GHC's git history remains linear ghc/ghc will use GitLab's > >> "fast-forward without a merge commit" merge strategy. > > > > Are merge requests squashed before they are merged? > > > > It seems that the answer by default is no.. > > https://gitlab.com/gitlab-org/gitlab-ce/issues/27956 > > > Indeed there are not. Moreover, in the workflow that I suggest in the > email not squashing is the desired behavior since each commit is atomic. > > > and the reason being that upsteam prefers "Convention over > > Configuration".. > > https://about.gitlab.com/handbook/product/#convention-over-configuration > > > > However it seems that there is a per-mr option which can be checked if > > you are diligent to do it for each MR. Some comments indicate that > > it's possible to implement a webhook to change this behaviour. > > > I'm not sure there is a reasonable default here; not matter what you > choose it will be wrong a good fraction of the time. The current plan is > to just ensure that the person who merges an MR considers whether the > history it introduces is useful and choose the correct option. I believe > this should work fine although I'm happy to reconsider if not. > > Cheers, > > - Ben > From ben at well-typed.com Thu Dec 27 16:18:00 2018 From: ben at well-typed.com (Ben Gamari) Date: Thu, 27 Dec 2018 11:18:00 -0500 Subject: [GHC DevOps Group] Welcome to GitLab! In-Reply-To: References: <87a7krscvf.fsf@smart-cactus.org> <871s63roms.fsf@smart-cactus.org> Message-ID: <87wonvq6z7.fsf@smart-cactus.org> Matthew Pickering writes: > Does this mean that the commit message doesn't come from the MR description? > Correct. Like GitHub, the commit messages that make it into the version control history are precisely the commit messages of the commits in your branch. > If I want to edit the commit message for a description I need to > > 1. squash the changes locally > 2. force push the branch > 3. Wait for CI to finish building (even though I just changed the > commit message). > This would be a great time to use the "Merge when CI passes" feature. Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From ben at well-typed.com Mon Dec 31 17:58:46 2018 From: ben at well-typed.com (Ben Gamari) Date: Mon, 31 Dec 2018 12:58:46 -0500 Subject: [GHC DevOps Group] Welcome to GitLab! In-Reply-To: <87wonvq6z7.fsf@smart-cactus.org> References: <87a7krscvf.fsf@smart-cactus.org> <871s63roms.fsf@smart-cactus.org> <87wonvq6z7.fsf@smart-cactus.org> Message-ID: <87d0phpoh8.fsf@smart-cactus.org> Ben Gamari writes: > Matthew Pickering writes: > >> Does this mean that the commit message doesn't come from the MR description? >> > Correct. Like GitHub, the commit messages that make it into the version > control history are precisely the commit messages of the commits in your > branch. > I should note that the ability to specify the commit message when squashing is an open feature request [1]. It's not currently prioritized but I will bring it up with upstream. Cheers, - Ben [1] https://gitlab.com/gitlab-org/gitlab-ee/issues/1510 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: