From lonetiger at gmail.com Sun Jan 6 18:17:53 2019 From: lonetiger at gmail.com (Phyx) Date: Sun, 6 Jan 2019 18:17:53 +0000 Subject: [GHC DevOps Group] CircleCI status In-Reply-To: <87efa7okjb.fsf@smart-cactus.org> References: <8736r2tjpm.fsf@smart-cactus.org> <87tvjgsvp4.fsf@smart-cactus.org> <87woo1o9b2.fsf@smart-cactus.org> <87r2e9o3xg.fsf@smart-cactus.org> <87o99dnza7.fsf@smart-cactus.org> <87efa7okjb.fsf@smart-cactus.org> Message-ID: Hi Ben, Sorry I had completely missed this email, gmail shuffled it away. I should have mentioned that majority of the changes are opt-in on that branch through a define _USE_NATIVE_PE_CHECKSUM I have a package build for it that I use here https://github.com/Mistuke/mingw-w64-packages I can build you a one if you want. Cheers, Tamar On Mon, Dec 24, 2018 at 12:06 AM Ben Gamari wrote: > Phyx writes: > > > Hi Ben, > > > > In case you haven't built one yet, here's a GCC 8 build with the patch > > back-ported. > > > > > https://mistuke.blob.core.windows.net/binaries/releases/mingw-w64-x86_64-gcc-libs-8.2.1+20181214-1-any.pkg.tar.xz > > > https://mistuke.blob.core.windows.net/binaries/releases/mingw-w64-x86_64-gcc-8.2.1+20181214-1-any.pkg.tar.xz > > > > I'm still trying to build lld but CMAKE is not playing along. Will let it > > build everything overnight. > > > Thanks Tamar! > > Unfortunately, it sounds [1] like lld won't be a workable solution for > quite a while as GHC apparently produces relocations which lld does not > yet support. I'm trying to build your binutils branch currently. > > Cheers, > > - Ben > > > [1] https://ghc.haskell.org/trac/ghc/ticket/16084#comment:5 > -------------- next part -------------- An HTML attachment was scrubbed... URL: From marlowsd at gmail.com Mon Jan 7 08:42:14 2019 From: marlowsd at gmail.com (Simon Marlow) Date: Mon, 7 Jan 2019 08:42:14 +0000 Subject: [GHC DevOps Group] Welcome to GitLab! In-Reply-To: <87a7krscvf.fsf@smart-cactus.org> References: <87a7krscvf.fsf@smart-cactus.org> Message-ID: Congrats Ben and co! This is a huge step forwards. On Thu, 27 Dec 2018 at 06:27, Ben Gamari wrote: > > git remote set-url origin https://gitlab.haskell.org/ghc/ghc.git > git remote set-url --push origin git at gitlab.haskell.org:ghc/ghc > > This is all that should be necessary; a quick `git pull origin master` > should verify that everything is working as expected. > submodules are still pulling from git.haskell.org, is there an easy way to fix that? Cheers Simon -------------- next part -------------- An HTML attachment was scrubbed... URL: From omeragacan at gmail.com Mon Jan 7 08:54:26 2019 From: omeragacan at gmail.com (=?UTF-8?Q?=C3=96mer_Sinan_A=C4=9Facan?=) Date: Mon, 7 Jan 2019 11:54:26 +0300 Subject: [GHC DevOps Group] Welcome to GitLab! In-Reply-To: References: <87a7krscvf.fsf@smart-cactus.org> Message-ID: > submodules are still pulling from git.haskell.org, is there an easy way to fix that? `git submodule sync` should fix that. Ömer Simon Marlow , 7 Oca 2019 Pzt, 11:42 tarihinde şunu yazdı: > > Congrats Ben and co! This is a huge step forwards. > > On Thu, 27 Dec 2018 at 06:27, Ben Gamari wrote: >> >> >> git remote set-url origin https://gitlab.haskell.org/ghc/ghc.git >> git remote set-url --push origin git at gitlab.haskell.org:ghc/ghc >> >> This is all that should be necessary; a quick `git pull origin master` >> should verify that everything is working as expected. > > > submodules are still pulling from git.haskell.org, is there an easy way to fix that? > > Cheers > Simon > _______________________________________________ > Ghc-devops-group mailing list > Ghc-devops-group at haskell.org > https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devops-group From ben at well-typed.com Mon Jan 7 21:28:01 2019 From: ben at well-typed.com (Ben Gamari) Date: Mon, 07 Jan 2019 16:28:01 -0500 Subject: [GHC DevOps Group] Welcome to GitLab! In-Reply-To: References: <87a7krscvf.fsf@smart-cactus.org> Message-ID: <87bm4sma3m.fsf@smart-cactus.org> Simon Marlow writes: > Congrats Ben and co! This is a huge step forwards. > Thanks! > submodules are still pulling from git.haskell.org, is there an easy way to > fix that? Indeed, Omer's advice is spot on. Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From ben at well-typed.com Tue Jan 8 01:57:03 2019 From: ben at well-typed.com (Ben Gamari) Date: Mon, 07 Jan 2019 20:57:03 -0500 Subject: [GHC DevOps Group] CircleCI status In-Reply-To: References: <8736r2tjpm.fsf@smart-cactus.org> <87tvjgsvp4.fsf@smart-cactus.org> <87woo1o9b2.fsf@smart-cactus.org> <87r2e9o3xg.fsf@smart-cactus.org> <87o99dnza7.fsf@smart-cactus.org> <87efa7okjb.fsf@smart-cactus.org> Message-ID: <87wonflxn7.fsf@smart-cactus.org> Phyx writes: > Hi Ben, > > Sorry I had completely missed this email, gmail shuffled it away. > > I should have mentioned that majority of the changes are opt-in on that > branch through a define _USE_NATIVE_PE_CHECKSUM > I have a package build for it that I use here > https://github.com/Mistuke/mingw-w64-packages > Ahhh! That explains why I saw such little change. I took a brief look at the diff but somehow missed this. > I can build you a one if you want. > That would be lovely. Thank you! Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From simonpj at microsoft.com Sun Jan 20 22:56:23 2019 From: simonpj at microsoft.com (Simon Peyton Jones) Date: Sun, 20 Jan 2019 22:56:23 +0000 Subject: [GHC DevOps Group] Guidelines In-Reply-To: <9AF21783-FA63-4D16-AD63-D4B024378885@tweag.io> References: <9AF21783-FA63-4D16-AD63-D4B024378885@tweag.io> Message-ID: Manuel Do you think sufficient time elapsed to deem silence as consent? Mind you, I doubt it's controversial, but I'd prefer members of the group to assent more explicitly. Simon | -----Original Message----- | From: Manuel Chakravarty | Sent: 06 December 2018 16:05 | To: ghc-devops-group at haskell.org | Cc: Simon Peyton Jones | Subject: Guidelines | | Dear DevOps Group, | | The GHC Steering Committee has recently decided to adopt a set of | ”Guidelines for Respectful Communication” due to ongoing concern about the | standards of discourse in the Haskell community: | | | https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.co | m%2Fghc-proposals%2Fghc- | proposals%2Fblob%2Fmaster%2FGRC.rst&data=02%7C01%7Csimonpj%40microsoft. | com%7Cbb8ab81f5d2049aa1d0508d65b949e3b%7C72f988bf86f141af91ab2d7cd011db47%7 | C1%7C0%7C636797091206572659&sdata=o9GqJGAt1QUF8ONgz8ASpBJj5%2Bq%2B3Ci4I | IabuWVor9Y%3D&reserved=0 | | I’ll leave it to Simon PJ’s excellent explanation to lay out the detailed | motivation: | | > As many of you will know, I’ve been concerned for several years about the | standards of discourse in the Haskell community. I think things have | improved since the period that drove me to write my Respect email | , but it’s far from secure. | > | > I have thought about trying to introduce a code of conduct for the | Haskell community, but (a) getting agreement about such a thing would be | hard, (b) we have no credible enforcement mechanism and (c) some people | disagree that it would even be helpful. | > | > I have also considered publishing a kind of personal code of conduct, | describing the standards to which I hold myself (doubtless all too | fallibly). I was concerned that this would over-personalise the issue. | > | > In the light of all this, following discussion at ICFP, the GHC Steering | Committee has decided to adopt these "Guidelines for Respectful | Communication”. | > | > as a middle way. We are adopting it for ourselves, not imposing it on | others. But in doing so we hope to give a clear lead to the community | about the standards that we espouse for our professional dialogue. | > | > We deliberately decided to use the term “guidelines for respectful | communication”, rather than “code of conduct”, for reasons that Stallman | lays out in his recently-released GNU Kind Communications Guidelines: | https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Flwn.net%2 | FArticles%2F769167%2F&data=02%7C01%7Csimonpj%40microsoft.com%7Cbb8ab81f | 5d2049aa1d0508d65b949e3b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C63679 | 7091206572659&sdata=Ac%2FPsmcP0T7AGBGfdb%2F9OKJbvb%2BnqNNDhfLAARk7daM%3 | D&reserved=0 | | In this context, it is important to note that the intention for these | guidelines is to apply to us (the DevOps group members) with the intention | of setting a good example and not to police the broader community, which we | do not have the mandate to do anyway. | | Personally, I am strongly in favour for us to follow the example of the GHC | Steering Committee and to adopt the same rules, in order to project a | uniform picture from all GHC committees. | | Cheers, | Manuel From manuel.chakravarty at tweag.io Mon Jan 21 20:25:27 2019 From: manuel.chakravarty at tweag.io (Manuel Chakravarty) Date: Mon, 21 Jan 2019 21:25:27 +0100 Subject: [GHC DevOps Group] Guidelines In-Reply-To: <9AF21783-FA63-4D16-AD63-D4B024378885@tweag.io> References: <9AF21783-FA63-4D16-AD63-D4B024378885@tweag.io> Message-ID: <0A7F0787-ED52-49A9-8C87-7A3BC5DBD106@tweag.io> More than a month elapsed since this message. Is the silence tacit approval or disinterest in the topic? I’d love to hear at least from some of you. Cheers, Manuel > Am 06.12.2018 um 17:05 schrieb Manuel Chakravarty : > > Dear DevOps Group, > > The GHC Steering Committee has recently decided to adopt a set of ”Guidelines for Respectful Communication” due to ongoing concern about the standards of discourse in the Haskell community: > > https://github.com/ghc-proposals/ghc-proposals/blob/master/GRC.rst > > I’ll leave it to Simon PJ’s excellent explanation to lay out the detailed motivation: > >> As many of you will know, I’ve been concerned for several years about the standards of discourse in the Haskell community. I think things have improved since the period that drove me to write my Respect email , but it’s far from secure. >> >> I have thought about trying to introduce a code of conduct for the Haskell community, but (a) getting agreement about such a thing would be hard, (b) we have no credible enforcement mechanism and (c) some people disagree that it would even be helpful. >> >> I have also considered publishing a kind of personal code of conduct, describing the standards to which I hold myself (doubtless all too fallibly). I was concerned that this would over-personalise the issue. >> >> In the light of all this, following discussion at ICFP, the GHC Steering Committee has decided to adopt these "Guidelines for Respectful Communication”. >> >> as a middle way. We are adopting it for ourselves, not imposing it on others. But in doing so we hope to give a clear lead to the community about the standards that we espouse for our professional dialogue. >> >> We deliberately decided to use the term “guidelines for respectful communication”, rather than “code of conduct”, for reasons that Stallman lays out in his recently-released GNU Kind Communications Guidelines: https://lwn.net/Articles/769167/ > > In this context, it is important to note that the intention for these guidelines is to apply to us (the DevOps group members) with the intention of setting a good example and not to police the broader community, which we do not have the mandate to do anyway. > > Personally, I am strongly in favour for us to follow the example of the GHC Steering Committee and to adopt the same rules, in order to project a uniform picture from all GHC committees. > > Cheers, > Manuel > > _______________________________________________ > Ghc-devops-group mailing list > Ghc-devops-group at haskell.org > https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devops-group From ben at well-typed.com Mon Jan 21 22:32:10 2019 From: ben at well-typed.com (Ben Gamari) Date: Mon, 21 Jan 2019 17:32:10 -0500 Subject: [GHC DevOps Group] Guidelines In-Reply-To: <0A7F0787-ED52-49A9-8C87-7A3BC5DBD106@tweag.io> References: <9AF21783-FA63-4D16-AD63-D4B024378885@tweag.io> <0A7F0787-ED52-49A9-8C87-7A3BC5DBD106@tweag.io> Message-ID: <87k1ixd4m2.fsf@smart-cactus.org> Manuel Chakravarty writes: > More than a month elapsed since this message. Is the silence tacit > approval or disinterest in the topic? I’d love to hear at least from > some of you. > I think that the steering committee was right in adopted these guidelines and believe that we should follow suit. Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From ben at well-typed.com Tue Jan 22 18:41:03 2019 From: ben at well-typed.com (Ben Gamari) Date: Tue, 22 Jan 2019 13:41:03 -0500 Subject: [GHC DevOps Group] Help with S3 costs Message-ID: <87y37cbkng.fsf@smart-cactus.org> Hi everyone, Our GitLab service currently uses Amazon S3 to back storage of artifacts and backups. At the moment we retain binary distributions for all builds for two weeks. This is proving to be a significant cost center, primarily due to transfer costs. For instance, our artifacts S3 bucket is currently around 400 GB and this month the transfer costs alone will be nearly 100 USD. Thankfully, the cost of the byte-hours is comparatively small, being less than 10 USD. Currently I am paying for this personally but admittedly the cost has been a bit higher than I had anticipated. I see a few avenues which I think we should pursue: * we can reduce the volume of builds that we preserve; I have started (!187) with the obvious step of only preserving builds from the `master` branch although we could probably pare down more. * we could just move artifact storage back to block storage on Packet.net. This storage is free to us and worked quite well before we switched to S3. We could keep backups on S3 to ensure they are in safe keeping. * someone on the ghc-devops group would be able to pick up the S3 tab. Thoughts? Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From ben at well-typed.com Tue Jan 22 19:57:31 2019 From: ben at well-typed.com (Ben Gamari) Date: Tue, 22 Jan 2019 14:57:31 -0500 Subject: [GHC DevOps Group] Help with S3 costs In-Reply-To: <87y37cbkng.fsf@smart-cactus.org> References: <87y37cbkng.fsf@smart-cactus.org> Message-ID: <87munsbh3r.fsf@smart-cactus.org> Ben Gamari writes: > Hi everyone, > > Our GitLab service currently uses Amazon S3 to back storage of > artifacts and backups. At the moment we retain binary distributions for > all builds for two weeks. This is proving to be a significant cost > center, primarily due to transfer costs. For instance, our artifacts S3 > bucket is currently around 400 GB and this month the transfer costs > alone will be nearly 100 USD. Thankfully, the cost of the byte-hours is > comparatively small, being less than 10 USD. > > Currently I am paying for this personally but admittedly the cost has > been a bit higher than I had anticipated. I see a few avenues which I > think we should pursue: > > * we can reduce the volume of builds that we preserve; I have started > (!187) with the obvious step of only preserving builds from the > `master` branch although we could probably pare down more. > > * we could just move artifact storage back to block storage on > Packet.net. This storage is free to us and worked quite well before > we switched to S3. We could keep backups on S3 to ensure they are in > safe keeping. > > * someone on the ghc-devops group would be able to pick up the S3 tab. > As it turns out there may be a fourth option: * Reduce our bandwidth requirements without sacrificing services. It turns out, this is quite feasible: a user in #ghc noticed that our previous bandwidth requirements were likely due to the fact that gitlab-runner was downloading artifacts far more often than necessary [1]. Fixing this is trivial (!188) and should nearly eliminate our transfer expenses. Of course, it would still be great if we could work out a way to cover this cost in the future. I'd imagine it will be on the order of 10 USD/month unless something changes. Cheers, - Ben [1] Specifically, the `dependencies` field defaults to a job depending on all of the jobs in the preceding stage (https://docs.gitlab.com/ee/ci/yaml/#dependencies). -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From simonpj at microsoft.com Tue Jan 22 22:50:12 2019 From: simonpj at microsoft.com (Simon Peyton Jones) Date: Tue, 22 Jan 2019 22:50:12 +0000 Subject: [GHC DevOps Group] Help with S3 costs In-Reply-To: <87y37cbkng.fsf@smart-cactus.org> References: <87y37cbkng.fsf@smart-cactus.org> Message-ID: I think it would probably help to summarise all the costs that we are currently incurring, and who is currently paying for them (including in-kind contributions). That would help to give perspective. Not incurring costs unnecessarily is obviously the first step, so your second message (with good news about reducing storage costs) is great. If the remaining monthly cost still looks high, we should review whether we want to keep everything we are keeping. Does anyone ever look at this stuff? You should not have to pay for anything personally! Simon | -----Original Message----- | From: Ghc-devops-group On Behalf Of | Ben Gamari | Sent: 22 January 2019 18:41 | To: ghc-devops-group at haskell.org | Subject: [GHC DevOps Group] Help with S3 costs | | Hi everyone, | | Our GitLab service currently uses Amazon S3 to back storage of | artifacts and backups. At the moment we retain binary distributions for | all builds for two weeks. This is proving to be a significant cost | center, primarily due to transfer costs. For instance, our artifacts S3 | bucket is currently around 400 GB and this month the transfer costs | alone will be nearly 100 USD. Thankfully, the cost of the byte-hours is | comparatively small, being less than 10 USD. | | Currently I am paying for this personally but admittedly the cost has | been a bit higher than I had anticipated. I see a few avenues which I | think we should pursue: | | * we can reduce the volume of builds that we preserve; I have started | (!187) with the obvious step of only preserving builds from the | `master` branch although we could probably pare down more. | | * we could just move artifact storage back to block storage on | Packet.net. This storage is free to us and worked quite well before | we switched to S3. We could keep backups on S3 to ensure they are in | safe keeping. | | * someone on the ghc-devops group would be able to pick up the S3 tab. | | Thoughts? | | Cheers, | | - Ben From ben at well-typed.com Tue Jan 22 23:38:32 2019 From: ben at well-typed.com (Ben Gamari) Date: Tue, 22 Jan 2019 18:38:32 -0500 Subject: [GHC DevOps Group] Help with S3 costs In-Reply-To: References: <87y37cbkng.fsf@smart-cactus.org> Message-ID: <87imyg9sb1.fsf@smart-cactus.org> Simon Peyton Jones writes: > I think it would probably help to summarise all the costs that we are > currently incurring, and who is currently paying for them (including > in-kind contributions). That would help to give perspective. > Yes, that is a great idea. From the top of my head here is a quick summary; currently all of our costs except for S3 are covered by in-kind donations (thank you everyone!): * Who: Google X What: Google Compute Engine credit * currently 4x 8-core x86-64/Windows instances Why: Currently used exclusively for Windows CI builds although we aren't using the entire credit Cost: In-kind donation * Who: Packet.net What: Computers * 2x 8-core x86-64/Linux machines * 1x 48-core AArch64/Linux machine Why: * GitLab hosting * CI builders * various non-GHC services (Hackage, Hackage documentation builder, haskell.org web hosting) Cost: In-kind donation * Who: Futurice What: Computers * 1x Mac Mini Why: * Mac OS X builder Cost: In-kind donation * Who: Davean Scies What: Computers * 2x x86-64/Darwin Mac Minis Why: * Mac OS X builders Cost: In-kind donation * Who: Ben Gamari What: Computers * 1x 16-core x86-64/Linux machine * 1x 64-core x86-64/Linux machine Why: * CI builders Cost: In-kind donation * Who: Alp Mestanogullari What: Computers * 1x 12-core x86-64/Linux machine Why: * CI builder Cost: In-kind donation * Who: GitLab What: A GitLab Ultimate license Why: * Hopefully this is self-explanatory Cost: In-kind donation I do hope I haven't forgotten anyone. > Not incurring costs unnecessarily is obviously the first step, so your > second message (with good news about reducing storage costs) is great. > If the remaining monthly cost still looks high, we should review > whether we want to keep everything we are keeping. Does anyone ever > look at this stuff? > Preserving binary distributions serves two purposes: * preserving `master` commits allows for easy bisection; I think this is quite important since bisection is frequently the fastest way to home in on a regression . * preserving MR artifacts allows users to easily use and share the results of their builders; given that we have burned the carbon to produce the build, it seems wasteful to immediately throw away the result given how cheap storage is. Facilitating this is the purpose of the ghc-artefact-nix tool which mpickering recently wrote about on ghc-devs. Now since we have avoided the worst of transfer costs it seems to me that the cost preserving these artifacts is well-justified. > You should not have to pay for anything personally! > Of course, to be clear I don't intend on continuing to pay the AWS costs; the bucket merely ended up on my account since this was the easiest way to get started. The plan was to then bring the issue with ghc-devops after we had a better idea of what the costs look like. Admittedly, I expected the bill to be a bit lower than it ended up being. Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From lonetiger at gmail.com Tue Jan 22 23:50:47 2019 From: lonetiger at gmail.com (Phyx) Date: Tue, 22 Jan 2019 23:50:47 +0000 Subject: [GHC DevOps Group] CircleCI status In-Reply-To: <87wonflxn7.fsf@smart-cactus.org> References: <8736r2tjpm.fsf@smart-cactus.org> <87tvjgsvp4.fsf@smart-cactus.org> <87woo1o9b2.fsf@smart-cactus.org> <87r2e9o3xg.fsf@smart-cactus.org> <87o99dnza7.fsf@smart-cactus.org> <87efa7okjb.fsf@smart-cactus.org> <87wonflxn7.fsf@smart-cactus.org> Message-ID: Hi Ben, Sorry for the delay, had been away. I've rebased the branch and spun a 64-bit tarball https://mistuke.blob.core.windows.net/binaries/releases/mingw-w64-x86_64-binutils-phyx.r6.1b62f9f9-1-any.pkg.tar.xz If this doesn't have any effect then I think I'd need to profile a slow linking example. The problem is that eventually the cache gets hot so I can't tell unless I flush it if it helped. I think dynamic linking should speed this up as well. I have that on my list after the I/O manager (which currently only lacks me finishing the new synchronization primitive, yay debugging CMM)... Let me know the results of your testing. I remember for my own testing with -ffunction-sections it did make difference. Cheers, Tamar On Tue, Jan 8, 2019 at 1:57 AM Ben Gamari wrote: > Phyx writes: > > > Hi Ben, > > > > Sorry I had completely missed this email, gmail shuffled it away. > > > > I should have mentioned that majority of the changes are opt-in on that > > branch through a define _USE_NATIVE_PE_CHECKSUM > > I have a package build for it that I use here > > https://github.com/Mistuke/mingw-w64-packages > > > Ahhh! That explains why I saw such little change. I took a brief look at > the diff but somehow missed this. > > > I can build you a one if you want. > > > That would be lovely. Thank you! > > Cheers, > > - Ben > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jonas.chevalier at tweag.io Mon Jan 28 17:18:37 2019 From: jonas.chevalier at tweag.io (Chevalier, Jonas) Date: Mon, 28 Jan 2019 18:18:37 +0100 Subject: [GHC DevOps Group] Help with S3 costs In-Reply-To: <87imyg9sb1.fsf@smart-cactus.org> References: <87y37cbkng.fsf@smart-cactus.org> <87imyg9sb1.fsf@smart-cactus.org> Message-ID: Hi Ben, Tweag is going to pick-up the tab for this. I will send you AWS credentials in a few. If all you want is a S3-compatible API, I have also deployed minio successfully in the past. There are a few gotchas but it could be a nice alternative. Best, Jonas On Wed, Jan 23, 2019 at 12:38 AM Ben Gamari wrote: > Simon Peyton Jones writes: > > > I think it would probably help to summarise all the costs that we are > > currently incurring, and who is currently paying for them (including > > in-kind contributions). That would help to give perspective. > > > Yes, that is a great idea. > > From the top of my head here is a quick summary; currently all of our > costs except for S3 are covered by in-kind donations (thank you > everyone!): > > * Who: Google X > What: Google Compute Engine credit > * currently 4x 8-core x86-64/Windows instances > Why: Currently used exclusively for Windows CI builds > although we aren't using the entire credit > Cost: In-kind donation > > * Who: Packet.net > What: Computers > * 2x 8-core x86-64/Linux machines > * 1x 48-core AArch64/Linux machine > Why: > * GitLab hosting > * CI builders > * various non-GHC services (Hackage, Hackage documentation > builder, haskell.org web hosting) > Cost: In-kind donation > > * Who: Futurice > What: Computers > * 1x Mac Mini > Why: > * Mac OS X builder > Cost: In-kind donation > > * Who: Davean Scies > What: Computers > * 2x x86-64/Darwin Mac Minis > Why: > * Mac OS X builders > Cost: In-kind donation > > * Who: Ben Gamari > What: Computers > * 1x 16-core x86-64/Linux machine > * 1x 64-core x86-64/Linux machine > Why: > * CI builders > Cost: In-kind donation > > * Who: Alp Mestanogullari > What: Computers > * 1x 12-core x86-64/Linux machine > Why: > * CI builder > Cost: In-kind donation > > * Who: GitLab > What: A GitLab Ultimate license > Why: > * Hopefully this is self-explanatory > Cost: In-kind donation > > I do hope I haven't forgotten anyone. > > > Not incurring costs unnecessarily is obviously the first step, so your > > second message (with good news about reducing storage costs) is great. > > If the remaining monthly cost still looks high, we should review > > whether we want to keep everything we are keeping. Does anyone ever > > look at this stuff? > > > Preserving binary distributions serves two purposes: > > * preserving `master` commits allows for easy bisection; I think this > is quite important since bisection is frequently the fastest way to > home in on a regression . > > * preserving MR artifacts allows users to easily use and share the > results of their builders; given that we have burned the carbon to > produce the build, it seems wasteful to immediately throw away the > result given how cheap storage is. Facilitating this is the purpose > of the ghc-artefact-nix tool which mpickering recently wrote about on > ghc-devs. > > Now since we have avoided the worst of transfer costs it seems to me > that the cost preserving these artifacts is well-justified. > > > You should not have to pay for anything personally! > > > Of course, to be clear I don't intend on continuing to pay the AWS > costs; the bucket merely ended up on my account since this was the > easiest way to get started. The plan was to then bring the issue with > ghc-devops after we had a better idea of what the costs look like. > Admittedly, I expected the bill to be a bit lower than it ended up > being. > > Cheers, > > - Ben > _______________________________________________ > Ghc-devops-group mailing list > Ghc-devops-group at haskell.org > https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devops-group > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben at well-typed.com Mon Jan 28 18:52:52 2019 From: ben at well-typed.com (Ben Gamari) Date: Mon, 28 Jan 2019 13:52:52 -0500 Subject: [GHC DevOps Group] Help with S3 costs In-Reply-To: References: <87y37cbkng.fsf@smart-cactus.org> <87imyg9sb1.fsf@smart-cactus.org> Message-ID: <87r2cwd380.fsf@smart-cactus.org> "Chevalier, Jonas" writes: > Hi Ben, > > Tweag is going to pick-up the tab for this. I will send you AWS credentials > in a few. > Thank you so much! When hvr noticed your email he recalled that Dreamobjects has apparently donated its services to haskell.org. They apparently expose an S3-compatible API so we could likely just move our traffic there as well. When it rains it pours! Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: