[GHC DevOps Group] Help with S3 costs

[Ben Gamari]

Simon Peyton Jones <simonpj at microsoft.com> writes:

> I think it would probably help to summarise all the costs that we are
> currently incurring, and who is currently paying for them (including
> in-kind contributions). That would help to give perspective.
>
Yes, that is a great idea.

From the top of my head here is a quick summary; currently all of our
costs except for S3 are covered by in-kind donations (thank you
everyone!):

 * Who: Google X
   What: Google Compute Engine credit
     * currently 4x 8-core x86-64/Windows instances
   Why: Currently used exclusively for Windows CI builds
        although we aren't using the entire credit
   Cost: In-kind donation

 * Who: Packet.net
   What: Computers
     * 2x 8-core x86-64/Linux machines
     * 1x 48-core AArch64/Linux machine
   Why:
     * GitLab hosting
     * CI builders
     * various non-GHC services (Hackage, Hackage documentation
       builder, haskell.org web hosting)
   Cost: In-kind donation

 * Who: Futurice
   What: Computers
     * 1x Mac Mini
   Why:
     * Mac OS X builder
   Cost: In-kind donation

 * Who: Davean Scies
   What: Computers
     * 2x x86-64/Darwin Mac Minis
   Why:
     * Mac OS X builders
   Cost: In-kind donation

 * Who: Ben Gamari
   What: Computers
     * 1x 16-core x86-64/Linux machine
     * 1x 64-core x86-64/Linux machine
   Why:
     * CI builders
   Cost: In-kind donation

 * Who: Alp Mestanogullari
   What: Computers
     * 1x 12-core x86-64/Linux machine
   Why:
     * CI builder
   Cost: In-kind donation

 * Who: GitLab
   What: A GitLab Ultimate license
   Why:
     * Hopefully this is self-explanatory
   Cost: In-kind donation

I do hope I haven't forgotten anyone.

> Not incurring costs unnecessarily is obviously the first step, so your
> second message (with good news about reducing storage costs) is great.
> If the remaining monthly cost still looks high, we should review
> whether we want to keep everything we are keeping. Does anyone ever
> look at this stuff?
>
Preserving binary distributions serves two purposes:

 * preserving `master` commits allows for easy bisection; I think this
   is quite important since bisection is frequently the fastest way to
   home in on a regression .

 * preserving MR artifacts allows users to easily use and share the
   results of their builders; given that we have burned the carbon to
   produce the build, it seems wasteful to immediately throw away the
   result given how cheap storage is. Facilitating this is the purpose
   of the ghc-artefact-nix tool which mpickering recently wrote about on
   ghc-devs.

Now since we have avoided the worst of transfer costs it seems to me
that the cost preserving these artifacts is well-justified.

> You should not have to pay for anything personally!
>
Of course, to be clear I don't intend on continuing to pay the AWS
costs; the bucket merely ended up on my account since this was the
easiest way to get started. The plan was to then bring the issue with
ghc-devops after we had a better idea of what the costs look like.
Admittedly, I expected the bill to be a bit lower than it ended up
being.

Cheers,

- Ben
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devops-group/attachments/20190122/80bc3534/attachment.sig>