[GHC DevOps Group] CircleCI status and budget concerns

[alex]

>  84% refers to our plan's container time allotment of 1500 hours

Even though Circle CI is using Google Cloud instances, the CI still imposes
a time limit? But why?

On 15 June 2018 at 19:13, Ben Gamari <ben at well-typed.com> wrote:

> Hi everyone,
>
> Over the last few weeks while finalizing the 8.6 branch I've been
> working on cleaning up some loose ends on the CI front. This includes,
>
>  * Fixing the Fedora and Hadrian builds
>  * Whittling away at the i386 Linux failures
>  * Incorporating nightly slow validation (which is now green, thanks to
>    Alp's efforts)
>  * Incorporating Alp's fixes to Hadrian to get this build green again
>
> During this process, however, I have noticed that a concerning banner
> has appeared at the top of the CircleCI interface. It currently reads:
>
>    Your current usage represents 84% of your ghc Linux plan's limit.
>    Please upgrade in order to ensure no disruption in building.
>
> With the percentage gradually ticking upwards as builds finish. I
> believe 84% refers to our plan's container time allotment of 1500
> hours (confusingly mislabeled as "1500 minutes" on the website). So far
> in the current May 21 - Jun 21 billing period we have used 1250 hours of
> build time. This is concerning as it presumably means that when we
> eventually overrun our budget we will be left without any CI at all for
> the remainder of the month.
>
> Given that we are currently building only a subset of commits on only
> the `master` I'm rather concerned that we are within a stone's throw of
> this situation (perhaps even going over this month). This seems
> untenable, especially when we consider that the number of builds will
> soon be growing to include stable branch backports, patches under
> review, and release builds.
>
> To manage this it seems that we will likely need to draw from a few of
> the tools at our disposal:
>
>  a. Reduce the number of build configurations that we build every commit
>     in (e.g. Fedora and LLVM could likely be relegated to a nightly build)
>
>  b. Reduce the amount of parallelism that we allow CircleCI to employ:
>     having access to CI consistently throughout the month is far
>     preferable to faster build turnarounds for some of the month
>     followed by periods of no CI for the rest.
>
>  c. Increase the build time limit; this is $50/month/container
>
> In the short-run we can implement (a) immediately. This of course will
> reduce our coverage, but it looks like our coverage goals may not be
> practical given the resources we have. I've looked into (b) and haven't
> found any indication that CircleCI supports our use-case. If this is
> the case then we will have to simply pursue (c).
>
> I suppose haskell.org may be able to contribute here. However, I wonder
> what other resources could be deployed.
>
> Cheers,
>
> - Ben
>
> _______________________________________________
> Ghc-devops-group mailing list
> Ghc-devops-group at haskell.org
> https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devops-group
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devops-group/attachments/20180615/5dd5259d/attachment.html>