GitLab runners having issues

[Bryan Richter]

Update: spurious failure tracking has been fixed!

On Wed, Nov 9, 2022 at 10:22 AM Bryan Richter <bryan at haskell.foundation> wrote:
>
> It looks like this was "resolved" -- it was an intermittent issue that
> went away on its own. I will start tracking it appropriately.
>
> Speaking of which, tracking spurious failures is not functioning right
> now, due to a problem with the ghc-perf-import bot. Until that gets
> resolved (later today, no doubt!), we'll see a higher pipeline failure
> rate.
>
> -Bryan
>
> On Tue, Nov 8, 2022 at 1:01 PM Bryan Richter <bryan at haskell.foundation> wrote:
> >
> > Hi everyone,
> >
> > Unfortunately, many CI jobs are failing for similar reasons on a
> > number of runners. Examples:
> >
> > 1.
> >
> > Running /usr/local/bin/cabal update --index=2020-12-21T14:48:20Z...
> > Downloading the latest package list from hackage.haskell.org
> > Version of <repo>/root.json is less than the previous version
> > /usr/local/bin/cabal update --index=2020-12-21T14:48:20Z failed
> > error: cabal update failed
> >
> > -- https://gitlab.haskell.org/ghc/ghc/-/jobs/1224391
> >
> >
> > 2.
> >
> > ERROR: Job failed: failed to pull image
> > "registry.gitlab.com/gitlab-org/gitlab-runner/gitlab-runner-helper:x86_64-v15.0.0"
> >
> > -- https://gitlab.haskell.org/ghc/ghc/-/jobs/1224641
> >
> > Moritz, can you look into these?
> >
> > Could it have something to do with the GitLab emergency upgrade that
> > happened yesterday? Or were the runners reconfigured in some way?
> >
> > I don't have administrative access to these runners.
> >
> > I have created an issue: https://gitlab.haskell.org/ghc/ghc/-/issues/22422
> >
> > -Bryan