[GHC DevOps Group] Help with S3 costs

[Chevalier, Jonas]

Hi Ben,

Tweag is going to pick-up the tab for this. I will send you AWS credentials
in a few.

If all you want is a S3-compatible API, I have also deployed minio
successfully in the past. There are a few gotchas but it could be a nice
alternative.

Best,
Jonas


On Wed, Jan 23, 2019 at 12:38 AM Ben Gamari <ben at well-typed.com> wrote:

> Simon Peyton Jones <simonpj at microsoft.com> writes:
>
> > I think it would probably help to summarise all the costs that we are
> > currently incurring, and who is currently paying for them (including
> > in-kind contributions). That would help to give perspective.
> >
> Yes, that is a great idea.
>
> From the top of my head here is a quick summary; currently all of our
> costs except for S3 are covered by in-kind donations (thank you
> everyone!):
>
>  * Who: Google X
>    What: Google Compute Engine credit
>      * currently 4x 8-core x86-64/Windows instances
>    Why: Currently used exclusively for Windows CI builds
>         although we aren't using the entire credit
>    Cost: In-kind donation
>
>  * Who: Packet.net
>    What: Computers
>      * 2x 8-core x86-64/Linux machines
>      * 1x 48-core AArch64/Linux machine
>    Why:
>      * GitLab hosting
>      * CI builders
>      * various non-GHC services (Hackage, Hackage documentation
>        builder, haskell.org web hosting)
>    Cost: In-kind donation
>
>  * Who: Futurice
>    What: Computers
>      * 1x Mac Mini
>    Why:
>      * Mac OS X builder
>    Cost: In-kind donation
>
>  * Who: Davean Scies
>    What: Computers
>      * 2x x86-64/Darwin Mac Minis
>    Why:
>      * Mac OS X builders
>    Cost: In-kind donation
>
>  * Who: Ben Gamari
>    What: Computers
>      * 1x 16-core x86-64/Linux machine
>      * 1x 64-core x86-64/Linux machine
>    Why:
>      * CI builders
>    Cost: In-kind donation
>
>  * Who: Alp Mestanogullari
>    What: Computers
>      * 1x 12-core x86-64/Linux machine
>    Why:
>      * CI builder
>    Cost: In-kind donation
>
>  * Who: GitLab
>    What: A GitLab Ultimate license
>    Why:
>      * Hopefully this is self-explanatory
>    Cost: In-kind donation
>
> I do hope I haven't forgotten anyone.
>
> > Not incurring costs unnecessarily is obviously the first step, so your
> > second message (with good news about reducing storage costs) is great.
> > If the remaining monthly cost still looks high, we should review
> > whether we want to keep everything we are keeping. Does anyone ever
> > look at this stuff?
> >
> Preserving binary distributions serves two purposes:
>
>  * preserving `master` commits allows for easy bisection; I think this
>    is quite important since bisection is frequently the fastest way to
>    home in on a regression .
>
>  * preserving MR artifacts allows users to easily use and share the
>    results of their builders; given that we have burned the carbon to
>    produce the build, it seems wasteful to immediately throw away the
>    result given how cheap storage is. Facilitating this is the purpose
>    of the ghc-artefact-nix tool which mpickering recently wrote about on
>    ghc-devs.
>
> Now since we have avoided the worst of transfer costs it seems to me
> that the cost preserving these artifacts is well-justified.
>
> > You should not have to pay for anything personally!
> >
> Of course, to be clear I don't intend on continuing to pay the AWS
> costs; the bucket merely ended up on my account since this was the
> easiest way to get started. The plan was to then bring the issue with
> ghc-devops after we had a better idea of what the costs look like.
> Admittedly, I expected the bill to be a bit lower than it ended up
> being.
>
> Cheers,
>
> - Ben
> _______________________________________________
> Ghc-devops-group mailing list
> Ghc-devops-group at haskell.org
> https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devops-group
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devops-group/attachments/20190128/16e51c47/attachment.html>