[GHC DevOps Group] Help with S3 costs

[Ben Gamari]

Hi everyone,

Our GitLab service currently uses Amazon S3 to back storage of
artifacts and backups. At the moment we retain binary distributions for
all builds for two weeks. This is proving to be a significant cost
center, primarily due to transfer costs. For instance, our artifacts S3
bucket is currently around 400 GB and this month the transfer costs
alone will be nearly 100 USD. Thankfully, the cost of the byte-hours is
comparatively small, being less than 10 USD.

Currently I am paying for this personally but admittedly the cost has
been a bit higher than I had anticipated. I see a few avenues which I
think we should pursue:

 * we can reduce the volume of builds that we preserve; I have started
   (!187) with the obvious step of only preserving builds from the
   `master` branch although we could probably pare down more.

 * we could just move artifact storage back to block storage on
   Packet.net. This storage is free to us and worked quite well before
   we switched to S3. We could keep backups on S3 to ensure they are in
   safe keeping.

 * someone on the ghc-devops group would be able to pick up the S3 tab.

Thoughts?

Cheers,

- Ben
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devops-group/attachments/20190122/f30f6cd2/attachment.sig>