<div dir="ltr"><div>I did find that some jobs were being retried repeatedly. I have deployed a workaround to prevent this from continuing. The problem was related to T18623 behaving strangely, and I've opened <a href="https://gitlab.haskell.org/ghc/ghc/-/issues/23139">https://gitlab.haskell.org/ghc/ghc/-/issues/23139</a> so GHC devs can have a look at it.</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sun, 19 Mar 2023 at 13:27, Bryan Richter <bryan@haskell.foundation> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>I'm back at my computer and am investigating. I don't see the problem I feared, but I do see some anomalies. I'll update once it's back to normal.<br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sat, 18 Mar 2023 at 17:55, Bryan Richter <bryan@haskell.foundation> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="auto">I'm away from my computer for the day, but yes there were some jobs that got stuck in a restart loop. See <a href="https://gitlab.haskell.org/ghc/ghc/-/issues/23094#note_487426" target="_blank">https://gitlab.haskell.org/ghc/ghc/-/issues/23094#note_487426</a> . Unfortunately I don't know if there are others, but I did fix the root cause of that particular loop.</div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sat, 18 Mar 2023, 15.06 Sam Derbyshire, <<a href="mailto:sam.derbyshire@gmail.com" target="_blank">sam.derbyshire@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>I think there's a problem with jobs restarting, on my <a href="https://gitlab.haskell.org/ghc/ghc/-/merge_requests/8686" rel="noreferrer" target="_blank">renamer MR</a> there were 5 full pipelines running at once. I had to cancel some of them, but also it seems some got cancelled by some new CI pipelines restarting.</div><div><br></div><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sat, 18 Mar 2023 at 13:59, Simon Peyton Jones <<a href="mailto:simon.peytonjones@gmail.com" rel="noreferrer" target="_blank">simon.peytonjones@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div style="font-family:tahoma,sans-serif">All GHC CI pipelines seem stalled, sadly</div><div style="font-family:tahoma,sans-serif"><br></div><div style="font-family:tahoma,sans-serif">e.g. <a href="https://gitlab.haskell.org/ghc/ghc/-/merge_requests/10123/pipelines" rel="noreferrer" target="_blank">https://gitlab.haskell.org/ghc/ghc/-/merge_requests/10123/pipelines</a></div><div style="font-family:tahoma,sans-serif"><br></div><div style="font-family:tahoma,sans-serif">Can someone unglue it?</div><div style="font-family:tahoma,sans-serif"><br></div><div style="font-family:tahoma,sans-serif">Thanks!</div><div style="font-family:tahoma,sans-serif"><br></div><div style="font-family:tahoma,sans-serif">Simon<br></div></div>
_______________________________________________<br>
ghc-devs mailing list<br>
<a href="mailto:ghc-devs@haskell.org" rel="noreferrer" target="_blank">ghc-devs@haskell.org</a><br>
<a href="http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs" rel="noreferrer noreferrer" target="_blank">http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs</a><br>
</blockquote></div></div>
</blockquote></div>
</blockquote></div>
</blockquote></div>