From ben at well-typed.com Mon Jul 1 17:22:42 2019 From: ben at well-typed.com (Ben Gamari) Date: Mon, 01 Jul 2019 13:22:42 -0400 Subject: [GHC DevOps Group] Making GHC's fast release cadence work Message-ID: <87a7dxwtyc.fsf@smart-cactus.org> Hi everyone, GHC's core libraries are a critical part of the Haskell ecosystem. I want to thank you for overseeing the maintenance of this infrastructure. However, for the last three weeks the release candidate for GHC 8.8.1 has been ready aside from releases of a couple of our core libraries. Naturally, delays like this make it hard for GHC to maintain its faster release cycle. At the same time, we do not want this cadence to impose an undue burden on our core library maintainers. How do you think we might speed up this process? For instance, perhaps the GHC release manager could pick up some of the "boring parts" of core library maintenance limited to: * Version bound bumps * Changes of CPP conditionals to accommodate changes in the compiler and other core libraries * Changelog entries to describe these releases * Uploading these releases or revisions to Hackage Of course, this would merely be an offer to maintainers; this would be GHC's way of carrying some of the burden that our release process imposes. In general, I am interested in a discussion on how to make this faster release pace work. Ideas? Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From david.feuer at gmail.com Mon Jul 1 17:30:57 2019 From: david.feuer at gmail.com (David Feuer) Date: Mon, 1 Jul 2019 13:30:57 -0400 Subject: [GHC DevOps Group] Making GHC's fast release cadence work In-Reply-To: <87a7dxwtyc.fsf@smart-cactus.org> References: <87a7dxwtyc.fsf@smart-cactus.org> Message-ID: I think it would be nice to start by letting the core library maintainers know your preferred release deadlines. On Mon, Jul 1, 2019, 1:22 PM Ben Gamari wrote: > Hi everyone, > > GHC's core libraries are a critical part of the Haskell ecosystem. I > want to thank you for overseeing the maintenance of this infrastructure. > > However, for the last three weeks the release candidate for GHC 8.8.1 > has been ready aside from releases of a couple of our core libraries. > > Naturally, delays like this make it hard for GHC to maintain its faster > release cycle. At the same time, we do not want this cadence to impose > an undue burden on our core library maintainers. > > How do you think we might speed up this process? > > For instance, perhaps the GHC release manager could pick up > some of the "boring parts" of core library maintenance limited to: > > * Version bound bumps > > * Changes of CPP conditionals to accommodate changes in the > compiler and other core libraries > > * Changelog entries to describe these releases > > * Uploading these releases or revisions to Hackage > > Of course, this would merely be an offer to maintainers; this would be > GHC's way of carrying some of the burden that our release process > imposes. > > In general, I am interested in a discussion on how to make this faster > release pace work. Ideas? > > Cheers, > > - Ben > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From d at davidterei.com Mon Jul 1 18:09:11 2019 From: d at davidterei.com (David Terei) Date: Mon, 1 Jul 2019 11:09:11 -0700 Subject: [GHC DevOps Group] Making GHC's fast release cadence work In-Reply-To: <87a7dxwtyc.fsf@smart-cactus.org> References: <87a7dxwtyc.fsf@smart-cactus.org> Message-ID: Ben, Everyone, Great timing! I noticed this the other day for the pretty library that I maintain. For me, the best solution would be to find a new maintainer. I can post on the general ghc/haskell mailing lists for someone who's interested, or if the core GHC team or anyone here could take this on please let me know. I've not been involved in the Haskell/GHC communities for a while now and sadly I don't see that changing anytime soon. My work on pretty is limited to life-support, I don't have the time to review and accept any code changes beyond trivial ones. Cheers, David On Mon, 1 Jul 2019 at 10:22, Ben Gamari wrote: > Hi everyone, > > GHC's core libraries are a critical part of the Haskell ecosystem. I > want to thank you for overseeing the maintenance of this infrastructure. > > However, for the last three weeks the release candidate for GHC 8.8.1 > has been ready aside from releases of a couple of our core libraries. > > Naturally, delays like this make it hard for GHC to maintain its faster > release cycle. At the same time, we do not want this cadence to impose > an undue burden on our core library maintainers. > > How do you think we might speed up this process? > > For instance, perhaps the GHC release manager could pick up > some of the "boring parts" of core library maintenance limited to: > > * Version bound bumps > > * Changes of CPP conditionals to accommodate changes in the > compiler and other core libraries > > * Changelog entries to describe these releases > > * Uploading these releases or revisions to Hackage > > Of course, this would merely be an offer to maintainers; this would be > GHC's way of carrying some of the burden that our release process > imposes. > > In general, I am interested in a discussion on how to make this faster > release pace work. Ideas? > > Cheers, > > - Ben > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.feuer at gmail.com Mon Jul 1 18:28:44 2019 From: david.feuer at gmail.com (David Feuer) Date: Mon, 1 Jul 2019 14:28:44 -0400 Subject: [GHC DevOps Group] Making GHC's fast release cadence work In-Reply-To: References: <87a7dxwtyc.fsf@smart-cactus.org> Message-ID: I realize that my response came off as rather unfriendly. I'm sorry about that. Thinking a bit more about it, I think GHC HQ can help Core Library maintainers in several unintrusive ways. Here are a couple I would appreciate myself; others may disagree. 1. Work with Herbert, Ryan, etc., to make it easy to use -Werror in Travis scripts. See https://github.com/haskell/containers/pull/645 2. Give us a nudge when you have a release timeframe in mind. "We're hoping to have the core libraries ready for release by December 5" (several weeks beforehand, repeated a couple times thereafter with appropriate adjustments) is much less stressful than "We're basically ready for release but we're waiting on core libraries." On Mon, Jul 1, 2019, 1:30 PM David Feuer wrote: > I think it would be nice to start by letting the core library maintainers > know your preferred release deadlines. > > On Mon, Jul 1, 2019, 1:22 PM Ben Gamari wrote: > >> Hi everyone, >> >> GHC's core libraries are a critical part of the Haskell ecosystem. I >> want to thank you for overseeing the maintenance of this infrastructure. >> >> However, for the last three weeks the release candidate for GHC 8.8.1 >> has been ready aside from releases of a couple of our core libraries. >> >> Naturally, delays like this make it hard for GHC to maintain its faster >> release cycle. At the same time, we do not want this cadence to impose >> an undue burden on our core library maintainers. >> >> How do you think we might speed up this process? >> >> For instance, perhaps the GHC release manager could pick up >> some of the "boring parts" of core library maintenance limited to: >> >> * Version bound bumps >> >> * Changes of CPP conditionals to accommodate changes in the >> compiler and other core libraries >> >> * Changelog entries to describe these releases >> >> * Uploading these releases or revisions to Hackage >> >> Of course, this would merely be an offer to maintainers; this would be >> GHC's way of carrying some of the burden that our release process >> imposes. >> >> In general, I am interested in a discussion on how to make this faster >> release pace work. Ideas? >> >> Cheers, >> >> - Ben >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben at well-typed.com Mon Jul 1 18:45:26 2019 From: ben at well-typed.com (Ben Gamari) Date: Mon, 01 Jul 2019 14:45:26 -0400 Subject: [GHC DevOps Group] Making GHC's fast release cadence work In-Reply-To: References: <87a7dxwtyc.fsf@smart-cactus.org> Message-ID: <874l45wq4d.fsf@smart-cactus.org> David Feuer writes: > I think it would be nice to start by letting the core library maintainers > know your preferred release deadlines. > Sure, that is reasonable. I do try to make the schedule clear in a few places: * At the beginning of the release cycle I send an email to all core library authors outlining the release cycle. This email was sent in late-October in the case of 8.8.1. * Per-project tracking tickets for the release * Individual reminders via email and IRC However, there no doubt could be more signalling throughout the process. In particular, I can be more communicative about changes in the GHC release process (e.g. letting core library maintainers know when we anticipate a schedule slip, as we did due to the GitLab migration). On the other hand, ideally GHC schedule changes wouldn't affect upstream release schedules. I think we would all probably be better off if core library upstreams try to keep releases for GHC small and boring. That is, preparation of a core library release would consist of taking the library's last major release, fixing any changed interfaces from dependencies, bumping some bounds, and, when GHC leaves alpha, pushing to Hackage. It's when a core library tries to synchronize its release cycle with GHC that we run into issues [1]. Cheers, - Ben [1] Yes, sometimes a synchronized release schedule is necessary, but I think we should avoid synchronization whenever possible. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From ben at well-typed.com Mon Jul 1 18:47:42 2019 From: ben at well-typed.com (Ben Gamari) Date: Mon, 01 Jul 2019 14:47:42 -0400 Subject: [GHC DevOps Group] Making GHC's fast release cadence work In-Reply-To: References: <87a7dxwtyc.fsf@smart-cactus.org> Message-ID: <871rz9wq0k.fsf@smart-cactus.org> David Terei writes: > Ben, Everyone, > > Great timing! I noticed this the other day for the pretty library that I > maintain. For me, the best solution would be to find a new maintainer. I > can post on the general ghc/haskell mailing lists for someone who's > interested, or if the core GHC team or anyone here could take this on > please let me know. > > I've not been involved in the Haskell/GHC communities for a while now and > sadly I don't see that changing anytime soon. My work on pretty is limited > to life-support, I don't have the time to review and accept any code > changes beyond trivial ones. > Hi David, Thanks for letting us know. Indeed I think asking around on the general Haskell lists is the right way forward. I would offer myself, but I'm afraid I already have more responsibilities than hours in the day. I hope you find more time for peace (and perhaps Haskell'ing) in the future. Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From m at tweag.io Mon Jul 1 20:51:24 2019 From: m at tweag.io (Boespflug, Mathieu) Date: Mon, 1 Jul 2019 22:51:24 +0200 Subject: [GHC DevOps Group] Making GHC's fast release cadence work In-Reply-To: References: <87a7dxwtyc.fsf@smart-cactus.org> Message-ID: A related discussion came up previously: https://mail.haskell.org/pipermail/ghc-devops-group/2017-December/thread.html#118. >From the moment that GHC accepts unreleased dependencies on its release branch, GHC developers lose full control over their own release timeline. Because a new GHC release can only ship when the dependencies have been released, the release process could block on upstream for indefinite periods of time, like is currently happening with at least one GHC dependency that has been blocked on the maintainer for at least 27 days (and counting). It was suggested then that GHC should only use release versions of its dependencies on the master or release branch, and that if this isn't practical due to co-dependency, then that suggests said dependencies should have the same release cycle as GHC (i.e. same people can cut releases of both) or even be merged in GHC itself. -- Mathieu Boespflug Founder at http://tweag.io. On Mon, 1 Jul 2019 at 21:03, David Feuer wrote: > I realize that my response came off as rather unfriendly. I'm sorry about > that. Thinking a bit more about it, I think GHC HQ can help Core Library > maintainers in several unintrusive ways. Here are a couple I would > appreciate myself; others may disagree. > > 1. Work with Herbert, Ryan, etc., to make it easy to use -Werror in Travis > scripts. See https://github.com/haskell/containers/pull/645 > > 2. Give us a nudge when you have a release timeframe in mind. "We're > hoping to have the core libraries ready for release by December 5" (several > weeks beforehand, repeated a couple times thereafter with appropriate > adjustments) is much less stressful than "We're basically ready for release > but we're waiting on core libraries." > > On Mon, Jul 1, 2019, 1:30 PM David Feuer wrote: > >> I think it would be nice to start by letting the core library maintainers >> know your preferred release deadlines. >> >> On Mon, Jul 1, 2019, 1:22 PM Ben Gamari wrote: >> >>> Hi everyone, >>> >>> GHC's core libraries are a critical part of the Haskell ecosystem. I >>> want to thank you for overseeing the maintenance of this infrastructure. >>> >>> However, for the last three weeks the release candidate for GHC 8.8.1 >>> has been ready aside from releases of a couple of our core libraries. >>> >>> Naturally, delays like this make it hard for GHC to maintain its faster >>> release cycle. At the same time, we do not want this cadence to impose >>> an undue burden on our core library maintainers. >>> >>> How do you think we might speed up this process? >>> >>> For instance, perhaps the GHC release manager could pick up >>> some of the "boring parts" of core library maintenance limited to: >>> >>> * Version bound bumps >>> >>> * Changes of CPP conditionals to accommodate changes in the >>> compiler and other core libraries >>> >>> * Changelog entries to describe these releases >>> >>> * Uploading these releases or revisions to Hackage >>> >>> Of course, this would merely be an offer to maintainers; this would be >>> GHC's way of carrying some of the burden that our release process >>> imposes. >>> >>> In general, I am interested in a discussion on how to make this faster >>> release pace work. Ideas? >>> >>> Cheers, >>> >>> - Ben >>> >>> _______________________________________________ > Ghc-devops-group mailing list > Ghc-devops-group at haskell.org > https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devops-group > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carter.schonwald at gmail.com Tue Jul 2 04:25:50 2019 From: carter.schonwald at gmail.com (Carter Schonwald) Date: Tue, 2 Jul 2019 00:25:50 -0400 Subject: [GHC DevOps Group] Making GHC's fast release cadence work In-Reply-To: References: <87a7dxwtyc.fsf@smart-cactus.org> Message-ID: what lib didn't have a release that wouldn't work with the release candidate? On Mon, Jul 1, 2019 at 4:51 PM Boespflug, Mathieu wrote: > A related discussion came up previously: > https://mail.haskell.org/pipermail/ghc-devops-group/2017-December/thread.html#118. > From the moment that GHC accepts unreleased dependencies on its release > branch, GHC developers lose full control over their own release timeline. > Because a new GHC release can only ship when the dependencies have been > released, the release process could block on upstream for indefinite > periods of time, like is currently happening with at least one GHC > dependency that has been blocked on the maintainer for at least 27 days > (and counting). > > It was suggested then that GHC should only use release versions of its > dependencies on the master or release branch, and that if this isn't > practical due to co-dependency, then that suggests said dependencies should > have the same release cycle as GHC (i.e. same people can cut releases of > both) or even be merged in GHC itself. > > -- > Mathieu Boespflug > Founder at http://tweag.io. > > > On Mon, 1 Jul 2019 at 21:03, David Feuer wrote: > >> I realize that my response came off as rather unfriendly. I'm sorry about >> that. Thinking a bit more about it, I think GHC HQ can help Core Library >> maintainers in several unintrusive ways. Here are a couple I would >> appreciate myself; others may disagree. >> >> 1. Work with Herbert, Ryan, etc., to make it easy to use -Werror in >> Travis scripts. See https://github.com/haskell/containers/pull/645 >> >> 2. Give us a nudge when you have a release timeframe in mind. "We're >> hoping to have the core libraries ready for release by December 5" (several >> weeks beforehand, repeated a couple times thereafter with appropriate >> adjustments) is much less stressful than "We're basically ready for release >> but we're waiting on core libraries." >> >> On Mon, Jul 1, 2019, 1:30 PM David Feuer wrote: >> >>> I think it would be nice to start by letting the core library >>> maintainers know your preferred release deadlines. >>> >>> On Mon, Jul 1, 2019, 1:22 PM Ben Gamari wrote: >>> >>>> Hi everyone, >>>> >>>> GHC's core libraries are a critical part of the Haskell ecosystem. I >>>> want to thank you for overseeing the maintenance of this infrastructure. >>>> >>>> However, for the last three weeks the release candidate for GHC 8.8.1 >>>> has been ready aside from releases of a couple of our core libraries. >>>> >>>> Naturally, delays like this make it hard for GHC to maintain its faster >>>> release cycle. At the same time, we do not want this cadence to impose >>>> an undue burden on our core library maintainers. >>>> >>>> How do you think we might speed up this process? >>>> >>>> For instance, perhaps the GHC release manager could pick up >>>> some of the "boring parts" of core library maintenance limited to: >>>> >>>> * Version bound bumps >>>> >>>> * Changes of CPP conditionals to accommodate changes in the >>>> compiler and other core libraries >>>> >>>> * Changelog entries to describe these releases >>>> >>>> * Uploading these releases or revisions to Hackage >>>> >>>> Of course, this would merely be an offer to maintainers; this would be >>>> GHC's way of carrying some of the burden that our release process >>>> imposes. >>>> >>>> In general, I am interested in a discussion on how to make this faster >>>> release pace work. Ideas? >>>> >>>> Cheers, >>>> >>>> - Ben >>>> >>>> _______________________________________________ >> Ghc-devops-group mailing list >> Ghc-devops-group at haskell.org >> https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devops-group >> > _______________________________________________ > Libraries mailing list > Libraries at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carter.schonwald at gmail.com Tue Jul 2 04:26:18 2019 From: carter.schonwald at gmail.com (Carter Schonwald) Date: Tue, 2 Jul 2019 00:26:18 -0400 Subject: [GHC DevOps Group] Making GHC's fast release cadence work In-Reply-To: References: <87a7dxwtyc.fsf@smart-cactus.org> Message-ID: err: which library lacked a release that worked with the 8.8 RC? On Tue, Jul 2, 2019 at 12:25 AM Carter Schonwald wrote: > what lib didn't have a release that wouldn't work with the release > candidate? > > > > On Mon, Jul 1, 2019 at 4:51 PM Boespflug, Mathieu wrote: > >> A related discussion came up previously: >> https://mail.haskell.org/pipermail/ghc-devops-group/2017-December/thread.html#118. >> From the moment that GHC accepts unreleased dependencies on its release >> branch, GHC developers lose full control over their own release timeline. >> Because a new GHC release can only ship when the dependencies have been >> released, the release process could block on upstream for indefinite >> periods of time, like is currently happening with at least one GHC >> dependency that has been blocked on the maintainer for at least 27 days >> (and counting). >> >> It was suggested then that GHC should only use release versions of its >> dependencies on the master or release branch, and that if this isn't >> practical due to co-dependency, then that suggests said dependencies should >> have the same release cycle as GHC (i.e. same people can cut releases of >> both) or even be merged in GHC itself. >> >> -- >> Mathieu Boespflug >> Founder at http://tweag.io. >> >> >> On Mon, 1 Jul 2019 at 21:03, David Feuer wrote: >> >>> I realize that my response came off as rather unfriendly. I'm sorry >>> about that. Thinking a bit more about it, I think GHC HQ can help Core >>> Library maintainers in several unintrusive ways. Here are a couple I would >>> appreciate myself; others may disagree. >>> >>> 1. Work with Herbert, Ryan, etc., to make it easy to use -Werror in >>> Travis scripts. See https://github.com/haskell/containers/pull/645 >>> >>> 2. Give us a nudge when you have a release timeframe in mind. "We're >>> hoping to have the core libraries ready for release by December 5" (several >>> weeks beforehand, repeated a couple times thereafter with appropriate >>> adjustments) is much less stressful than "We're basically ready for release >>> but we're waiting on core libraries." >>> >>> On Mon, Jul 1, 2019, 1:30 PM David Feuer wrote: >>> >>>> I think it would be nice to start by letting the core library >>>> maintainers know your preferred release deadlines. >>>> >>>> On Mon, Jul 1, 2019, 1:22 PM Ben Gamari wrote: >>>> >>>>> Hi everyone, >>>>> >>>>> GHC's core libraries are a critical part of the Haskell ecosystem. I >>>>> want to thank you for overseeing the maintenance of this >>>>> infrastructure. >>>>> >>>>> However, for the last three weeks the release candidate for GHC 8.8.1 >>>>> has been ready aside from releases of a couple of our core libraries. >>>>> >>>>> Naturally, delays like this make it hard for GHC to maintain its faster >>>>> release cycle. At the same time, we do not want this cadence to impose >>>>> an undue burden on our core library maintainers. >>>>> >>>>> How do you think we might speed up this process? >>>>> >>>>> For instance, perhaps the GHC release manager could pick up >>>>> some of the "boring parts" of core library maintenance limited to: >>>>> >>>>> * Version bound bumps >>>>> >>>>> * Changes of CPP conditionals to accommodate changes in the >>>>> compiler and other core libraries >>>>> >>>>> * Changelog entries to describe these releases >>>>> >>>>> * Uploading these releases or revisions to Hackage >>>>> >>>>> Of course, this would merely be an offer to maintainers; this would be >>>>> GHC's way of carrying some of the burden that our release process >>>>> imposes. >>>>> >>>>> In general, I am interested in a discussion on how to make this faster >>>>> release pace work. Ideas? >>>>> >>>>> Cheers, >>>>> >>>>> - Ben >>>>> >>>>> _______________________________________________ >>> Ghc-devops-group mailing list >>> Ghc-devops-group at haskell.org >>> https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devops-group >>> >> _______________________________________________ >> Libraries mailing list >> Libraries at haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hvriedel at gmail.com Mon Jul 1 21:12:42 2019 From: hvriedel at gmail.com (Herbert Valerio Riedel) Date: Mon, 1 Jul 2019 23:12:42 +0200 Subject: [GHC DevOps Group] Making GHC's fast release cadence work In-Reply-To: <87a7dxwtyc.fsf@smart-cactus.org> References: <87a7dxwtyc.fsf@smart-cactus.org> Message-ID: On Mon, Jul 1, 2019 at 8:09 PM Ben Gamari wrote: > Naturally, delays like this make it hard for GHC to maintain its faster release cycle > ... > How do you think we might speed up this process? IMO You're asking the wrong question. This seems based on a premise that everyone agreed with a faster release cycle... to me the downsides on the ecosystem and infrastructure of a faster release churn significantly outweight the modest benefit some people might perceive; and the issue's we've been observing (not only boot libraries, but also the 10k packages on Hackage) with the overspeeded release cycle are IMO a sign that we're moving faster than the ecosystem can accommodate. Just because GHC HQ managed to optimize their release processes (and I have to say, at the expense of the release quality -- GHC 8.6.5 was the first major release since GHC's beginning to require five attempts -- and I have to note that this results in annoying busy work for GHC packagers like myself) doesn't mean that everyone else has the time and energy to adapt to this new order as well. From ashley at semantic.org Mon Jul 1 20:08:14 2019 From: ashley at semantic.org (Ashley Yakeley) Date: Mon, 01 Jul 2019 13:08:14 -0700 Subject: [GHC DevOps Group] Making GHC's fast release cadence work In-Reply-To: <87a7dxwtyc.fsf@smart-cactus.org> References: <87a7dxwtyc.fsf@smart-cactus.org> Message-ID: <38e4a2588f684333566c1aa68efbc30236e2ee94.camel@semantic.org> So for the time library, these are my current understandings: 1. GHC will use the "ghc" branch of the source code, which I generally keep matched to the latest version of the library in Hackage. 2. If GHC HQ (or anyone) has a problem with the library, they will create an issue (or possibly a PR) in the library GitHub. 3. As the maintainer, I will fix library issues that block GHC releases (possibly with the help of GHC HQ). 4. If there are no issues that need to be fixed, library releases and GHC releases do not need to be synchronised. -- Ashley Yakeley On Mon, 2019-07-01 at 13:22 -0400, Ben Gamari wrote: > Hi everyone, > > GHC's core libraries are a critical part of the Haskell ecosystem. I > want to thank you for overseeing the maintenance of this > infrastructure. > > However, for the last three weeks the release candidate for GHC 8.8.1 > has been ready aside from releases of a couple of our core libraries. > > Naturally, delays like this make it hard for GHC to maintain its > faster > release cycle. At the same time, we do not want this cadence to > impose > an undue burden on our core library maintainers. > > How do you think we might speed up this process? > > For instance, perhaps the GHC release manager could pick up > some of the "boring parts" of core library maintenance limited to: > > * Version bound bumps > > * Changes of CPP conditionals to accommodate changes in the > compiler and other core libraries > > * Changelog entries to describe these releases > > * Uploading these releases or revisions to Hackage > > Of course, this would merely be an offer to maintainers; this would > be > GHC's way of carrying some of the burden that our release process > imposes. > > In general, I am interested in a discussion on how to make this > faster > release pace work. Ideas? > > Cheers, > > - Ben > -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael at fpcomplete.com Tue Jul 2 04:54:27 2019 From: michael at fpcomplete.com (Michael Snoyman) Date: Tue, 2 Jul 2019 07:54:27 +0300 Subject: [GHC DevOps Group] Making GHC's fast release cadence work In-Reply-To: References: <87a7dxwtyc.fsf@smart-cactus.org> Message-ID: On Tue, Jul 2, 2019 at 12:12 AM Herbert Valerio Riedel wrote: > On Mon, Jul 1, 2019 at 8:09 PM Ben Gamari wrote: > > Naturally, delays like this make it hard for GHC to maintain its faster > release cycle > > ... > > How do you think we might speed up this process? > > IMO You're asking the wrong question. > > This seems based on a premise that everyone agreed with a faster > release cycle... to me the downsides on the ecosystem and > infrastructure of a faster release churn significantly outweight the > modest benefit some people might perceive; and the issue's we've been > observing (not only boot libraries, but also the 10k packages on > Hackage) with the overspeeded release cycle are IMO a sign that we're > moving faster than the ecosystem can accommodate. Just because GHC HQ > managed to optimize their release processes (and I have to say, at the > expense of the release quality -- GHC 8.6.5 was the first major > release since GHC's beginning to require five attempts -- and I have > to note that this results in annoying busy work for GHC packagers like > myself) doesn't mean that everyone else has the time and energy to > adapt to this new order as well. > +1 -------------- next part -------------- An HTML attachment was scrubbed... URL: From simonpj at microsoft.com Tue Jul 2 12:15:40 2019 From: simonpj at microsoft.com (Simon Peyton Jones) Date: Tue, 2 Jul 2019 12:15:40 +0000 Subject: [GHC DevOps Group] Making GHC's fast release cadence work In-Reply-To: References: <87a7dxwtyc.fsf@smart-cactus.org> Message-ID: It seems to me that both questions are sensible ones: 1. What should GHC’s release cadence be? For example, would annual be better than semi-annual? Herbert and Michael both think so, and they know a thing or two. I’d love to hear from more people on this question. 1. When we do make a release, whenever that is, how we can work most productively with library authors, so that the release comes out in a timely way? This was Ben’s original question, and it applies regardless of cadence. Simon From: Ghc-devops-group On Behalf Of Michael Snoyman Sent: 02 July 2019 05:54 To: Herbert Valerio Riedel Cc: David Feuer ; ghc-devops-group at haskell.org; Ashley Yakeley ; Judah Jacobson ; David Terei ; Herbert Valerio Riedel ; Haskell Libraries (libraries at haskell.org) Subject: Re: [GHC DevOps Group] Making GHC's fast release cadence work On Tue, Jul 2, 2019 at 12:12 AM Herbert Valerio Riedel > wrote: On Mon, Jul 1, 2019 at 8:09 PM Ben Gamari > wrote: > Naturally, delays like this make it hard for GHC to maintain its faster release cycle > ... > How do you think we might speed up this process? IMO You're asking the wrong question. This seems based on a premise that everyone agreed with a faster release cycle... to me the downsides on the ecosystem and infrastructure of a faster release churn significantly outweight the modest benefit some people might perceive; and the issue's we've been observing (not only boot libraries, but also the 10k packages on Hackage) with the overspeeded release cycle are IMO a sign that we're moving faster than the ecosystem can accommodate. Just because GHC HQ managed to optimize their release processes (and I have to say, at the expense of the release quality -- GHC 8.6.5 was the first major release since GHC's beginning to require five attempts -- and I have to note that this results in annoying busy work for GHC packagers like myself) doesn't mean that everyone else has the time and energy to adapt to this new order as well. +1 -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthewtpickering at gmail.com Tue Jul 2 12:26:34 2019 From: matthewtpickering at gmail.com (Matthew Pickering) Date: Tue, 2 Jul 2019 13:26:34 +0100 Subject: [GHC DevOps Group] Making GHC's fast release cadence work In-Reply-To: References: <87a7dxwtyc.fsf@smart-cactus.org> Message-ID: Six months is even quite a long time between releases. What do other projects do? IMO, the main cause of this churn is the very incremental painful refactoring of the core type classes. Are there any other changes which regularly require library intervention? Cheers, Matt On Tue, Jul 2, 2019 at 1:16 PM Simon Peyton Jones via Libraries wrote: > > It seems to me that both questions are sensible ones: > > > > What should GHC’s release cadence be? For example, would annual be better than semi-annual? Herbert and Michael both think so, and they know a thing or two. I’d love to hear from more people on this question. > > > > When we do make a release, whenever that is, how we can work most productively with library authors, so that the release comes out in a timely way? This was Ben’s original question, and it applies regardless of cadence. > > > > Simon > > > > From: Ghc-devops-group On Behalf Of Michael Snoyman > Sent: 02 July 2019 05:54 > To: Herbert Valerio Riedel > Cc: David Feuer ; ghc-devops-group at haskell.org; Ashley Yakeley ; Judah Jacobson ; David Terei ; Herbert Valerio Riedel ; Haskell Libraries (libraries at haskell.org) > Subject: Re: [GHC DevOps Group] Making GHC's fast release cadence work > > > > > > > > On Tue, Jul 2, 2019 at 12:12 AM Herbert Valerio Riedel wrote: > > On Mon, Jul 1, 2019 at 8:09 PM Ben Gamari wrote: > > Naturally, delays like this make it hard for GHC to maintain its faster > release cycle > > ... > > How do you think we might speed up this process? > > IMO You're asking the wrong question. > > This seems based on a premise that everyone agreed with a faster > release cycle... to me the downsides on the ecosystem and > infrastructure of a faster release churn significantly outweight the > modest benefit some people might perceive; and the issue's we've been > observing (not only boot libraries, but also the 10k packages on > Hackage) with the overspeeded release cycle are IMO a sign that we're > moving faster than the ecosystem can accommodate. Just because GHC HQ > managed to optimize their release processes (and I have to say, at the > expense of the release quality -- GHC 8.6.5 was the first major > release since GHC's beginning to require five attempts -- and I have > to note that this results in annoying busy work for GHC packagers like > myself) doesn't mean that everyone else has the time and energy to > adapt to this new order as well. > > > > +1 > > _______________________________________________ > Libraries mailing list > Libraries at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries From lemming at henning-thielemann.de Tue Jul 2 12:36:19 2019 From: lemming at henning-thielemann.de (Henning Thielemann) Date: Tue, 2 Jul 2019 14:36:19 +0200 (CEST) Subject: [GHC DevOps Group] Making GHC's fast release cadence work In-Reply-To: References: <87a7dxwtyc.fsf@smart-cactus.org> Message-ID: On Tue, 2 Jul 2019, Simon Peyton Jones via Libraries wrote: > 1. What should GHC’s release cadence be?  For example, would annual be > better than semi-annual?  Herbert and > Michael both think so, and they know a thing or two.   I’d love to > hear from more people on this question. Annual is better for me. I do not need the latest and greatest GHC features immediately. I am happy to get bugfixes quickly and easily via hvr's ghc ppa. However, I'd prefer that libraries are separated from GHC and 'base' would be split. From ben at well-typed.com Tue Jul 2 14:55:45 2019 From: ben at well-typed.com (Ben Gamari) Date: Tue, 02 Jul 2019 10:55:45 -0400 Subject: [GHC DevOps Group] Making GHC's fast release cadence work In-Reply-To: References: <87a7dxwtyc.fsf@smart-cactus.org> Message-ID: <874l44v637.fsf@smart-cactus.org> Herbert Valerio Riedel writes: > On Mon, Jul 1, 2019 at 8:09 PM Ben Gamari wrote: >> Naturally, delays like this make it hard for GHC to maintain its faster > release cycle >> ... >> How do you think we might speed up this process? > > IMO You're asking the wrong question. > > This seems based on a premise that everyone agreed with a faster > release cycle... to me the downsides on the ecosystem and > infrastructure of a faster release churn significantly outweight the > modest benefit some people might perceive; and the issue's we've been > observing (not only boot libraries, but also the 10k packages on > Hackage) with the overspeeded release cycle are IMO a sign that we're > moving faster than the ecosystem can accommodate. As Simon said, I am happy to discuss both questions. However, I think the two matters are largely orthogonal. The timing of core library releases has been a problem even prior to the faster cadence. Consequently, in the interest of keeping this thread focused let's address the question of cadence elsewhere. Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From simonpj at microsoft.com Tue Jul 2 16:18:43 2019 From: simonpj at microsoft.com (Simon Peyton Jones) Date: Tue, 2 Jul 2019 16:18:43 +0000 Subject: [GHC DevOps Group] Making GHC's fast release cadence work In-Reply-To: <6f89f2fb-32b1-0976-ee76-f0213f029ea2@obsidian.systems> References: <87a7dxwtyc.fsf@smart-cactus.org> <6f89f2fb-32b1-0976-ee76-f0213f029ea2@obsidian.systems> Message-ID: | - the kicker: Any library that wishes to be included in GHC / base *must | share maintainership* with the GHC release team / libraries committee, | respectively/./ That means those groups can release things whenever they | want to unblock themselves. That seems very plausible to me. But "sharing maintainership" could be limited to: The GHC release team is free to make a minor (patch-level) release of an existing released version of the library L, embodying any changes necessary to make the library work with the new version of GHC I think that is all that's needed. In general GHC should not rely on a bleeding-edge release of a library. But it's not uncommon for tiny changes to be needed, and by sharing maintainership, the library author would not need to be bothered about making and releasing those changes. It's worth noting that it's not so much "libraries that want to be included in GHC", but rather "libraries that GHC's authors would like to import in GHC's own source code, rather than re-implementing that functionality (which would be silly)". Simon | -----Original Message----- | From: John Ericson | Sent: 02 July 2019 17:00 | To: Henning Thielemann ; Simon Peyton Jones | | Cc: ghc-devops-group at haskell.org; Michael Snoyman ; | David Terei ; Ashley Yakeley ; | Herbert Valerio Riedel ; Haskell Libraries | (libraries at haskell.org) | Subject: Re: [GHC DevOps Group] Making GHC's fast release cadence work | | Yeah there are many interlocking issues here. I've been thinking about | writing one of those ecosystem proposals about this. Here's some | observations about the status quo for now. | | - Frequent releases are a really nice for feature for the community. I know | I, for one, would be less motivated to work on GHC if it would take a year | in some case for my contributions to trickle back to my day job. | Rust releases every 6 *weeks*. I see no reason why we shouldn't aim for | that. | | - The GitLab move and switch to CI have made GHC development more | accessible and, temporary Marge Bot issues aside, better scaling. I am | immensely grateful for those that put in the work for that every MR I | write. Going back to annual releases would feel like a regression, | cancelling out some of those gains. | | - Base really is a PVP mess. Only the messy guts actually make sense to | pair with GHC releases, a breaking change every time. The "pure" bits would | naturally have *fewer* breaking releases, but *more* non-breaking releases. | | - AMP-like proposals could probably move faster if they weren't tied to GHC | releases. I.e. if multiple GHC releases supported multiple class | hierarchies, people could move at their own pace to some extent, and we | could support faster *and* slower migrations. | | - I hope GHC can move in a more modular direction, which by definition | means farming off work to libraries. In other words, I think GHC should | ultimately have the same degree of code reuse as any other Haskell project. | But the current experience with submodules is excruciating, so no one | dealing with GHC every day rightfully has any appetite for that. | It's a pity because once it's done I think it would be *less* work for the | core GHC team, there's just the awkward transition out of today's | monolithic local maximum. | | So all this needs leads me to conclude a few things | | - Base must be broken up, so it can be released more often than GHC. | Base would continue to exist reexport code in other libraries exclusively. | New users, executables, and legacy code can continue to depend on base as | before, but libraries are encouraged to adopt more fine-grained | dependencies on the underlying libraries. | | - the kicker: Any library that wishes to be included in GHC / base *must | share maintainership* with the GHC release team / libraries committee, | respectively/./ That means those groups can release things whenever they | want to unblock themselves. | // | | It's fairly clear-cut to me: having your library being included in GHC or | base is a big honor, and so you should not be able to hold up the release | process of these important projects unilaterally. If you want to keep full | custody of your baby, that's fine, but it's worth not the precious time of | the GHC/base developers to use it then. [Aim for, say, Haskell Platform | inclusion rather than base inclusion.] | | Granted, GHC, unlike Base, there is the alternative to use Cabal's new | support for multiple versions of a package in the same package DB. But | until will have private dependencies that would place some probably | unpopular constraints on consumers of GHC-the-library. If GHC does migrate | from ~10 to ~100 dependencies, maybe it might make sense to float that as | an alternative, but seeing that is many GHC releases away no matter what, I | think it's best to start with the shared maintainership policy. | |  John | | On 7/2/19 8:36 AM, Henning Thielemann wrote: | > | > On Tue, 2 Jul 2019, Simon Peyton Jones via Libraries wrote: | > | >>  1. What should GHC’s release cadence be? For example, would annual be | >>     better than semi-annual?  Herbert and | >>     Michael both think so, and they know a thing or two.   I’d love to | >>     hear from more people on this question. | > | > Annual is better for me. I do not need the latest and greatest GHC | > features immediately. I am happy to get bugfixes quickly and easily | > via hvr's ghc ppa. | > | > However, I'd prefer that libraries are separated from GHC and 'base' | > would be split. | > | > _______________________________________________ | > Libraries mailing list | > Libraries at haskell.org | > | https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.haske | ll.org%2Fcgi- | bin%2Fmailman%2Flistinfo%2Flibraries&data=02%7C01%7Csimonpj%40microsoft | .com%7C6ae01104c4b345984a0208d6ff06464e%7C72f988bf86f141af91ab2d7cd011db47% | 7C1%7C1%7C636976799775801107&sdata=YEf6lczTdY9Mp%2B6lFHzCdwcagiQWZ3VgxI | QTfqH5ZXk%3D&reserved=0 From david.feuer at gmail.com Tue Jul 2 16:29:30 2019 From: david.feuer at gmail.com (David Feuer) Date: Tue, 2 Jul 2019 12:29:30 -0400 Subject: [GHC DevOps Group] Making GHC's fast release cadence work In-Reply-To: References: <87a7dxwtyc.fsf@smart-cactus.org> <6f89f2fb-32b1-0976-ee76-f0213f029ea2@obsidian.systems> Message-ID: The biggest problems are for packages like containers, that are not only used by GHC but also exposed to users through the GHC API. These libraries aren't part of GHC or base, but pretty much have to move in lock step. On Tue, Jul 2, 2019, 12:19 PM Simon Peyton Jones via Libraries < libraries at haskell.org> wrote: > | - the kicker: Any library that wishes to be included in GHC / base *must > | share maintainership* with the GHC release team / libraries committee, > | respectively/./ That means those groups can release things whenever they > | want to unblock themselves. > > That seems very plausible to me. But "sharing maintainership" could be > limited to: > > The GHC release team is free to make a minor (patch-level) > release of an existing released version of the library L, > embodying any changes necessary to make the library work > with the new version of GHC > > I think that is all that's needed. In general GHC should not rely > on a bleeding-edge release of a library. But it's not uncommon > for tiny changes to be needed, and by sharing maintainership, the > library author would not need to be bothered about making and > releasing those changes. > > It's worth noting that it's not so much "libraries that want to > be included in GHC", but rather "libraries that GHC's authors > would like to import in GHC's own source code, rather than > re-implementing that functionality (which would be silly)". > > Simon > > | -----Original Message----- > | From: John Ericson > | Sent: 02 July 2019 17:00 > | To: Henning Thielemann ; Simon Peyton > Jones > | > | Cc: ghc-devops-group at haskell.org; Michael Snoyman < > michael at fpcomplete.com>; > | David Terei ; Ashley Yakeley ; > | Herbert Valerio Riedel ; Haskell Libraries > | (libraries at haskell.org) > | Subject: Re: [GHC DevOps Group] Making GHC's fast release cadence work > | > | Yeah there are many interlocking issues here. I've been thinking about > | writing one of those ecosystem proposals about this. Here's some > | observations about the status quo for now. > | > | - Frequent releases are a really nice for feature for the community. I > know > | I, for one, would be less motivated to work on GHC if it would take a > year > | in some case for my contributions to trickle back to my day job. > | Rust releases every 6 *weeks*. I see no reason why we shouldn't aim for > | that. > | > | - The GitLab move and switch to CI have made GHC development more > | accessible and, temporary Marge Bot issues aside, better scaling. I am > | immensely grateful for those that put in the work for that every MR I > | write. Going back to annual releases would feel like a regression, > | cancelling out some of those gains. > | > | - Base really is a PVP mess. Only the messy guts actually make sense to > | pair with GHC releases, a breaking change every time. The "pure" bits > would > | naturally have *fewer* breaking releases, but *more* non-breaking > releases. > | > | - AMP-like proposals could probably move faster if they weren't tied to > GHC > | releases. I.e. if multiple GHC releases supported multiple class > | hierarchies, people could move at their own pace to some extent, and we > | could support faster *and* slower migrations. > | > | - I hope GHC can move in a more modular direction, which by definition > | means farming off work to libraries. In other words, I think GHC should > | ultimately have the same degree of code reuse as any other Haskell > project. > | But the current experience with submodules is excruciating, so no one > | dealing with GHC every day rightfully has any appetite for that. > | It's a pity because once it's done I think it would be *less* work for > the > | core GHC team, there's just the awkward transition out of today's > | monolithic local maximum. > | > | So all this needs leads me to conclude a few things > | > | - Base must be broken up, so it can be released more often than GHC. > | Base would continue to exist reexport code in other libraries > exclusively. > | New users, executables, and legacy code can continue to depend on base as > | before, but libraries are encouraged to adopt more fine-grained > | dependencies on the underlying libraries. > | > | - the kicker: Any library that wishes to be included in GHC / base *must > | share maintainership* with the GHC release team / libraries committee, > | respectively/./ That means those groups can release things whenever they > | want to unblock themselves. > | // > | > | It's fairly clear-cut to me: having your library being included in GHC or > | base is a big honor, and so you should not be able to hold up the release > | process of these important projects unilaterally. If you want to keep > full > | custody of your baby, that's fine, but it's worth not the precious time > of > | the GHC/base developers to use it then. [Aim for, say, Haskell Platform > | inclusion rather than base inclusion.] > | > | Granted, GHC, unlike Base, there is the alternative to use Cabal's new > | support for multiple versions of a package in the same package DB. But > | until will have private dependencies that would place some probably > | unpopular constraints on consumers of GHC-the-library. If GHC does > migrate > | from ~10 to ~100 dependencies, maybe it might make sense to float that as > | an alternative, but seeing that is many GHC releases away no matter > what, I > | think it's best to start with the shared maintainership policy. > | > | John > | > | On 7/2/19 8:36 AM, Henning Thielemann wrote: > | > > | > On Tue, 2 Jul 2019, Simon Peyton Jones via Libraries wrote: > | > > | >> 1. What should GHC’s release cadence be? For example, would annual be > | >> better than semi-annual? Herbert and > | >> Michael both think so, and they know a thing or two. I’d love to > | >> hear from more people on this question. > | > > | > Annual is better for me. I do not need the latest and greatest GHC > | > features immediately. I am happy to get bugfixes quickly and easily > | > via hvr's ghc ppa. > | > > | > However, I'd prefer that libraries are separated from GHC and 'base' > | > would be split. > | > > | > _______________________________________________ > | > Libraries mailing list > | > Libraries at haskell.org > | > > | > https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.haske > | ll.org%2Fcgi- > | > bin%2Fmailman%2Flistinfo%2Flibraries&data=02%7C01%7Csimonpj%40microsoft > | > .com%7C6ae01104c4b345984a0208d6ff06464e%7C72f988bf86f141af91ab2d7cd011db47% > | > 7C1%7C1%7C636976799775801107&sdata=YEf6lczTdY9Mp%2B6lFHzCdwcagiQWZ3VgxI > | QTfqH5ZXk%3D&reserved=0 > _______________________________________________ > Libraries mailing list > Libraries at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ashley at semantic.org Tue Jul 2 16:32:08 2019 From: ashley at semantic.org (Ashley Yakeley) Date: Tue, 02 Jul 2019 09:32:08 -0700 Subject: [GHC DevOps Group] Making GHC's fast release cadence work In-Reply-To: References: <87a7dxwtyc.fsf@smart-cactus.org> <6f89f2fb-32b1-0976-ee76-f0213f029ea2@obsidian.systems> Message-ID: <498def267f3aba2f8e299a244f732f64d06980b7.camel@semantic.org> For the time library, I would rather the GHC team make a pull request first, and let me do the release. At the very least I want to know what's going on and what the needs are. -- Ashley On Tue, 2019-07-02 at 16:18 +0000, Simon Peyton Jones wrote: > The GHC release team is free to make a minor (patch-level) > release of an existing released version of the library L, > embodying any changes necessary to make the library work > with the new version of GHC > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben at smart-cactus.org Tue Jul 2 16:54:10 2019 From: ben at smart-cactus.org (Ben Gamari) Date: Tue, 02 Jul 2019 12:54:10 -0400 Subject: [GHC DevOps Group] Making GHC's fast release cadence work In-Reply-To: References: <87a7dxwtyc.fsf@smart-cactus.org> <6f89f2fb-32b1-0976-ee76-f0213f029ea2@obsidian.systems> Message-ID: <87o92ctm1b.fsf@smart-cactus.org> David Feuer writes: > The biggest problems are for packages like containers, that are not only > used by GHC but also exposed to users through the GHC API. These libraries > aren't part of GHC or base, but pretty much have to move in lock step. > I'm not sure I understand why this is so. Yes, install plans involving the GHC library are forced to use the same version of containers that GHC uses, but I would think that this is not the common case. Assuming most people aren't linking against the GHC library then I don't see the harm in GHC staying a bit behind upstreams like containers. Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From ben at smart-cactus.org Tue Jul 2 16:55:38 2019 From: ben at smart-cactus.org (Ben Gamari) Date: Tue, 02 Jul 2019 12:55:38 -0400 Subject: [GHC DevOps Group] Making GHC's fast release cadence work In-Reply-To: <498def267f3aba2f8e299a244f732f64d06980b7.camel@semantic.org> References: <87a7dxwtyc.fsf@smart-cactus.org> <6f89f2fb-32b1-0976-ee76-f0213f029ea2@obsidian.systems> <498def267f3aba2f8e299a244f732f64d06980b7.camel@semantic.org> Message-ID: <87lfxgtlyw.fsf@smart-cactus.org> Ashley Yakeley writes: > For the time library, I would rather the GHC team make a pull request > first, and let me do the release. At the very least I want to know > what's going on and what the needs are. > As long as the review turnaround is reasonably fast (which it always has been in the case of time; thank you!), I am perfectly content with this. Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From david.feuer at gmail.com Tue Jul 2 16:58:10 2019 From: david.feuer at gmail.com (David Feuer) Date: Tue, 2 Jul 2019 12:58:10 -0400 Subject: [GHC DevOps Group] Making GHC's fast release cadence work In-Reply-To: <87o92ctm1b.fsf@smart-cactus.org> References: <87a7dxwtyc.fsf@smart-cactus.org> <6f89f2fb-32b1-0976-ee76-f0213f029ea2@obsidian.systems> <87o92ctm1b.fsf@smart-cactus.org> Message-ID: It seems to mostly be an issue for stack, which wants a fully consistent package set. On Tue, Jul 2, 2019, 12:54 PM Ben Gamari wrote: > David Feuer writes: > > > The biggest problems are for packages like containers, that are not only > > used by GHC but also exposed to users through the GHC API. These > libraries > > aren't part of GHC or base, but pretty much have to move in lock step. > > > I'm not sure I understand why this is so. Yes, install plans involving > the GHC library are forced to use the same version of containers that > GHC uses, but I would think that this is not the common case. > > Assuming most people aren't linking against the GHC library then I don't > see the harm in GHC staying a bit behind upstreams like containers. > > Cheers, > > - Ben > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From john.ericson at obsidian.systems Tue Jul 2 15:59:32 2019 From: john.ericson at obsidian.systems (John Ericson) Date: Tue, 2 Jul 2019 11:59:32 -0400 Subject: [GHC DevOps Group] Making GHC's fast release cadence work In-Reply-To: References: <87a7dxwtyc.fsf@smart-cactus.org> Message-ID: <6f89f2fb-32b1-0976-ee76-f0213f029ea2@obsidian.systems> Yeah there are many interlocking issues here. I've been thinking about writing one of those ecosystem proposals about this. Here's some observations about the status quo for now. - Frequent releases are a really nice for feature for the community. I know I, for one, would be less motivated to work on GHC if it would take a year in some case for my contributions to trickle back to my day job. Rust releases every 6 *weeks*. I see no reason why we shouldn't aim for that. - The GitLab move and switch to CI have made GHC development more accessible and, temporary Marge Bot issues aside, better scaling. I am immensely grateful for those that put in the work for that every MR I write. Going back to annual releases would feel like a regression, cancelling out some of those gains. - Base really is a PVP mess. Only the messy guts actually make sense to pair with GHC releases, a breaking change every time. The "pure" bits would naturally have *fewer* breaking releases, but *more* non-breaking releases. - AMP-like proposals could probably move faster if they weren't tied to GHC releases. I.e. if multiple GHC releases supported multiple class hierarchies, people could move at their own pace to some extent, and we could support faster *and* slower migrations. - I hope GHC can move in a more modular direction, which by definition means farming off work to libraries. In other words, I think GHC should ultimately have the same degree of code reuse as any other Haskell project. But the current experience with submodules is excruciating, so no one dealing with GHC every day rightfully has any appetite for that. It's a pity because once it's done I think it would be *less* work for the core GHC team, there's just the awkward transition out of today's monolithic local maximum. So all this needs leads me to conclude a few things - Base must be broken up, so it can be released more often than GHC. Base would continue to exist reexport code in other libraries exclusively. New users, executables, and legacy code can continue to depend on base as before, but libraries are encouraged to adopt more fine-grained dependencies on the underlying libraries. - the kicker: Any library that wishes to be included in GHC / base *must share maintainership* with the GHC release team / libraries committee, respectively/./ That means those groups can release things whenever they want to unblock themselves. // It's fairly clear-cut to me: having your library being included in GHC or base is a big honor, and so you should not be able to hold up the release process of these important projects unilaterally. If you want to keep full custody of your baby, that's fine, but it's worth not the precious time of the GHC/base developers to use it then. [Aim for, say, Haskell Platform inclusion rather than base inclusion.] Granted, GHC, unlike Base, there is the alternative to use Cabal's new support for multiple versions of a package in the same package DB. But until will have private dependencies that would place some probably unpopular constraints on consumers of GHC-the-library. If GHC does migrate from ~10 to ~100 dependencies, maybe it might make sense to float that as an alternative, but seeing that is many GHC releases away no matter what, I think it's best to start with the shared maintainership policy.  John On 7/2/19 8:36 AM, Henning Thielemann wrote: > > On Tue, 2 Jul 2019, Simon Peyton Jones via Libraries wrote: > >>  1. What should GHC’s release cadence be? For example, would annual be >>     better than semi-annual?  Herbert and >>     Michael both think so, and they know a thing or two.   I’d love to >>     hear from more people on this question. > > Annual is better for me. I do not need the latest and greatest GHC > features immediately. I am happy to get bugfixes quickly and easily > via hvr's ghc ppa. > > However, I'd prefer that libraries are separated from GHC and 'base' > would be split. > > _______________________________________________ > Libraries mailing list > Libraries at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries From ben at smart-cactus.org Tue Jul 2 16:46:08 2019 From: ben at smart-cactus.org (Ben Gamari) Date: Tue, 02 Jul 2019 12:46:08 -0400 Subject: [GHC DevOps Group] Making GHC's fast release cadence work In-Reply-To: References: <87a7dxwtyc.fsf@smart-cactus.org> <6f89f2fb-32b1-0976-ee76-f0213f029ea2@obsidian.systems> Message-ID: <87sgrotmet.fsf@smart-cactus.org> Simon Peyton Jones via Libraries writes: > | - the kicker: Any library that wishes to be included in GHC / base *must > | share maintainership* with the GHC release team / libraries committee, > | respectively/./ That means those groups can release things whenever they > | want to unblock themselves. > > That seems very plausible to me. But "sharing maintainership" could be limited to: > > The GHC release team is free to make a minor (patch-level) > release of an existing released version of the library L, > embodying any changes necessary to make the library work > with the new version of GHC > > I think that is all that's needed. In general GHC should not rely > on a bleeding-edge release of a library. But it's not uncommon > for tiny changes to be needed, and by sharing maintainership, the > library author would not need to be bothered about making and > releasing those changes. > Yes, this is precisely what I was trying to articulate in my original email (although failed to do so clearly). We would not want to be responsible for managing major releases; merely incremental patch-level releases to patch things up to be shippable with GHC. Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From john.ericson at obsidian.systems Tue Jul 2 17:14:17 2019 From: john.ericson at obsidian.systems (John Ericson) Date: Tue, 2 Jul 2019 13:14:17 -0400 Subject: [GHC DevOps Group] Making GHC's fast release cadence work In-Reply-To: <87sgrotmet.fsf@smart-cactus.org> References: <87a7dxwtyc.fsf@smart-cactus.org> <6f89f2fb-32b1-0976-ee76-f0213f029ea2@obsidian.systems> <87sgrotmet.fsf@smart-cactus.org> Message-ID: <03e493e2-cc17-790b-9095-1c4f88bc8dd0@obsidian.systems> > The GHC release team is free to make a minor (patch-level) release of > an existing released version of the library L, embodying any changes > necessary to make the library work with the new version of GHC Yeah I like that limitation. It is good to make my proposal less draconian :) Happy to hear that's what you wanted all along Ben! Ashley: > For the time library, I would rather the GHC team make a pull request > first, and let me do the release. At the very least I want to know > what's going on and what the needs are. Adding to what Ben said, I think the way submodules work today mean that unmerged (e.g. just in the GHC fork) changes to a submodule block a PR? So only in repos where the PR happens to live in the same repo as master can GHC use the PR branch and still pass CI, and the usual case is the PR must be merged before GHC relies on it. I hope in the majority of cases these GHC-driven releases would just take stuff that is on master but unreleased, and therefore already reviewed the normal way. David: > It seems to mostly be an issue for stack, which wants a fully > consistent package set. Stackage should be comfortable building GHC itself with a newer containers then, I think. (And likewise for anyone that needs a newer containers in their build of the GHC library.) John On 7/2/19 12:46 PM, Ben Gamari wrote: > Simon Peyton Jones via Libraries writes: > >> | - the kicker: Any library that wishes to be included in GHC / base *must >> | share maintainership* with the GHC release team / libraries committee, >> | respectively/./ That means those groups can release things whenever they >> | want to unblock themselves. >> >> That seems very plausible to me. But "sharing maintainership" could be limited to: >> >> >> >> I think that is all that's needed. In general GHC should not rely >> on a bleeding-edge release of a library. But it's not uncommon >> for tiny changes to be needed, and by sharing maintainership, the >> library author would not need to be bothered about making and >> releasing those changes. >> > Yes, this is precisely what I was trying to articulate in my original > email (although failed to do so clearly). We would not want to be > responsible for managing major releases; merely incremental patch-level > releases to patch things up to be shippable with GHC. > > Cheers, > > - Ben > From ashley at semantic.org Tue Jul 2 21:39:31 2019 From: ashley at semantic.org (Ashley Yakeley) Date: Tue, 02 Jul 2019 14:39:31 -0700 Subject: [GHC DevOps Group] Making GHC's fast release cadence work In-Reply-To: <87a7dxwtyc.fsf@smart-cactus.org> References: <87a7dxwtyc.fsf@smart-cactus.org> Message-ID: Thinking about this, it seems like the root cause of the problem is that the relationship between GHC and the core libraries is awkwardly caught between two different models. In one model, the libraries are part of GHC, so the library maintainers are part of the GHC release team, that happen to be assigned these particular GHC features. If one member of the team can't work on their GHC feature for whatever reason, another member can step in. In the other model, the libraries are dependencies of GHC. GHC can make requests upstream to the libraries, but will have to wait or work around any issues in the mean time. -- Ashley On Mon, 2019-07-01 at 13:22 -0400, Ben Gamari wrote: > Hi everyone, > > GHC's core libraries are a critical part of the Haskell ecosystem. I > want to thank you for overseeing the maintenance of this > infrastructure. > > However, for the last three weeks the release candidate for GHC 8.8.1 > has been ready aside from releases of a couple of our core libraries. > > Naturally, delays like this make it hard for GHC to maintain its > faster > release cycle. At the same time, we do not want this cadence to > impose > an undue burden on our core library maintainers. > > How do you think we might speed up this process? > > For instance, perhaps the GHC release manager could pick up > some of the "boring parts" of core library maintenance limited to: > > * Version bound bumps > > * Changes of CPP conditionals to accommodate changes in the > compiler and other core libraries > > * Changelog entries to describe these releases > > * Uploading these releases or revisions to Hackage > > Of course, this would merely be an offer to maintainers; this would > be > GHC's way of carrying some of the burden that our release process > imposes. > > In general, I am interested in a discussion on how to make this > faster > release pace work. Ideas? > > Cheers, > > - Ben > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lemming at henning-thielemann.de Sat Jul 6 06:21:33 2019 From: lemming at henning-thielemann.de (Henning Thielemann) Date: Sat, 6 Jul 2019 08:21:33 +0200 (CEST) Subject: [GHC DevOps Group] Making GHC's fast release cadence work In-Reply-To: <6f89f2fb-32b1-0976-ee76-f0213f029ea2@obsidian.systems> References: <87a7dxwtyc.fsf@smart-cactus.org> <6f89f2fb-32b1-0976-ee76-f0213f029ea2@obsidian.systems> Message-ID: On Tue, 2 Jul 2019, John Ericson wrote: > Yeah there are many interlocking issues here. I've been thinking about > writing one of those ecosystem proposals about this. Here's some observations > about the status quo for now. > > - Frequent releases are a really nice for feature for the community. I know > I, for one, would be less motivated to work on GHC if it would take a year in > some case for my contributions to trickle back to my day job. Rust releases > every 6 *weeks*. I see no reason why we shouldn't aim for that. In the past we had nightly builds. They are not quite releases but if provided as, say, Debian packages it would be easy to install and upgrade them. From ben at well-typed.com Sat Jul 6 13:34:40 2019 From: ben at well-typed.com (Ben Gamari) Date: Sat, 06 Jul 2019 09:34:40 -0400 Subject: [GHC DevOps Group] Making GHC's fast release cadence work In-Reply-To: References: <87a7dxwtyc.fsf@smart-cactus.org> <6f89f2fb-32b1-0976-ee76-f0213f029ea2@obsidian.systems> Message-ID: <3C059EBF-E447-4E4C-9AB9-13AC946D00F0@well-typed.com> For what it's worth, we do provide nightly builds as a natural result of our CI infrastructure. Perhaps we should draw more attention to this fact. Cheers, Ben On July 6, 2019 2:21:33 AM EDT, Henning Thielemann wrote: > >On Tue, 2 Jul 2019, John Ericson wrote: > >> Yeah there are many interlocking issues here. I've been thinking >about >> writing one of those ecosystem proposals about this. Here's some >observations >> about the status quo for now. >> >> - Frequent releases are a really nice for feature for the community. >I know >> I, for one, would be less motivated to work on GHC if it would take a >year in >> some case for my contributions to trickle back to my day job. Rust >releases >> every 6 *weeks*. I see no reason why we shouldn't aim for that. > >In the past we had nightly builds. They are not quite releases but if >provided as, say, Debian packages it would be easy to install and >upgrade >them. >_______________________________________________ >Ghc-devops-group mailing list >Ghc-devops-group at haskell.org >https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devops-group -- Sent from my Android device with K-9 Mail. Please excuse my brevity. -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric at seidel.io Tue Jul 9 04:43:15 2019 From: eric at seidel.io (Eric Seidel) Date: Tue, 09 Jul 2019 00:43:15 -0400 Subject: [GHC DevOps Group] Making GHC's fast release cadence work In-Reply-To: References: <87a7dxwtyc.fsf@smart-cactus.org> <6f89f2fb-32b1-0976-ee76-f0213f029ea2@obsidian.systems> <87o92ctm1b.fsf@smart-cactus.org> Message-ID: <4a552f63-0441-4288-8196-c054da36871a@www.fastmail.com> Not just stack, system package managers want fully consistent package sets too. This is something I have to deal with occasionally when I do updates to our internal Haskell package set at work. On Tue, Jul 2, 2019, at 12:58, David Feuer wrote: > It seems to mostly be an issue for stack, which wants a fully > consistent package set. > > On Tue, Jul 2, 2019, 12:54 PM Ben Gamari wrote: > > David Feuer writes: > > > > > The biggest problems are for packages like containers, that are not only > > > used by GHC but also exposed to users through the GHC API. These libraries > > > aren't part of GHC or base, but pretty much have to move in lock step. > > > > > I'm not sure I understand why this is so. Yes, install plans involving > > the GHC library are forced to use the same version of containers that > > GHC uses, but I would think that this is not the common case. > > > > Assuming most people aren't linking against the GHC library then I don't > > see the harm in GHC staying a bit behind upstreams like containers. > > > > Cheers, > > > > - Ben > > > _______________________________________________ > Libraries mailing list > Libraries at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries > From hvriedel at gmail.com Tue Jul 9 08:29:44 2019 From: hvriedel at gmail.com (Herbert Valerio Riedel) Date: Tue, 9 Jul 2019 10:29:44 +0200 Subject: [GHC DevOps Group] Making GHC's fast release cadence work In-Reply-To: References: <87a7dxwtyc.fsf@smart-cactus.org> <6f89f2fb-32b1-0976-ee76-f0213f029ea2@obsidian.systems> Message-ID: > The biggest problems are for packages like containers, that are not only used by GHC but also exposed to users through the GHC API. These libraries aren't part of GHC or base, but pretty much have to move in lock step. fwiw, this what https://mail.haskell.org/pipermail/ghc-devs/2017-July/014424.html aims to address and would help to reduce this artificial boot-library version pinning for consumers. Moritz has been pushing this effort further via https://gitlab.haskell.org/ghc/ghc/merge_requests/490 but it didn't make it into GHC 8.8 yet. From moritz.angermann at gmail.com Tue Jul 9 12:36:32 2019 From: moritz.angermann at gmail.com (Moritz Angermann) Date: Tue, 9 Jul 2019 20:36:32 +0800 Subject: [GHC DevOps Group] Making GHC's fast release cadence work In-Reply-To: References: <87a7dxwtyc.fsf@smart-cactus.org> <6f89f2fb-32b1-0976-ee76-f0213f029ea2@obsidian.systems> Message-ID: > Moritz has been pushing this effort further via https://gitlab.haskell.org/ghc/ghc/merge_requests/490 but it didn't make it into GHC 8.8 yet. @John Ericson has been working on making ghc multi-target, which makes this much easier. So I'll rebase once that work is done. BUT: a big issue I've run into with a reinstallable lib:ghc is that it essentially breaks anything that somehow tied to ghc: Plugins, doctest, ... as you end up with two instances of `ghc` (one in ghc (stock lib:ghc), the other one (reinstalled lib:ghc) in the library you are linking). Now I had naively hoped this could *just* work out, but it didn't. On Tue, Jul 9, 2019 at 4:29 PM Herbert Valerio Riedel wrote: > > > The biggest problems are for packages like containers, that are not only used by GHC but also exposed to users through the GHC API. These libraries aren't part of GHC or base, but pretty much have to move in lock step. > > fwiw, this what > https://mail.haskell.org/pipermail/ghc-devs/2017-July/014424.html aims > to address and would help to reduce this artificial boot-library > version pinning for consumers. Moritz has been pushing this effort > further via https://gitlab.haskell.org/ghc/ghc/merge_requests/490 but > it didn't make it into GHC 8.8 yet. From nvazou at cs.ucsd.edu Fri Jul 26 12:11:24 2019 From: nvazou at cs.ucsd.edu (Niki Vazou) Date: Fri, 26 Jul 2019 14:11:24 +0200 Subject: [GHC DevOps Group] Haskell Implementors Working: Looking for Panelists In-Reply-To: References: Message-ID: Hello, Haskell Implementors Workshop 2019 will be having a 45 min panel discussion (17.15-18.00 on Fri 23th August). The goal is to have panelists that represent the following committees: - ghc steering committee - Devops committee - core libraries committee - Haskell' committee - Haskell.org committee - Haskell Symposium & HiW steering committee Let me know if you - want like to be in the panel - if there are issues you want to raise in the panel discussion. Best, Niki Chair of HiW'19 -------------- next part -------------- An HTML attachment was scrubbed... URL: