Consistent CI failure in job nightly-i386-linux-deb9-validate
Cheng Shao
cheng.shao at tweag.io
Wed Sep 28 11:02:10 UTC 2022
Sure, in which case pls revert it. Apologies for the impact, though
I'm still a bit curious, the i386 job did pass in the original MR.
On Wed, Sep 28, 2022 at 1:00 PM Bryan Richter <bryan at haskell.foundation> wrote:
>
> Yep, it seems to mostly be xz that is running out of memory. (All recent builds that I sampled, but not all builds through all time.) Thanks for pointing it out!
>
> I can revert the change.
>
> On Wed, Sep 28, 2022 at 11:46 AM Cheng Shao <cheng.shao at tweag.io> wrote:
>>
>> Hi Bryan,
>>
>> This may be an unintended fallout of !8940. Would you try starting an
>> i386 pipeline with it reversed to see if it solves the issue, in which
>> case we should revert or fix it in master?
>>
>> On Wed, Sep 28, 2022 at 9:58 AM Bryan Richter via ghc-devs
>> <ghc-devs at haskell.org> wrote:
>> >
>> > Hi all,
>> >
>> > For the past week or so, nightly-i386-linux-deb9-validate has been failing consistently.
>> >
>> > They show up on the failure dashboard because the logs contain the phrase "Cannot allocate memory".
>> >
>> > I haven't looked yet to see if they always fail in the same place, but I'll do that soon. The first example I looked at, however, has the line "xz: (stdin): Cannot allocate memory", so it's not GHC (alone) causing the problem.
>> >
>> > As a consequence of showing up on the dashboard, the jobs get restarted. Since they fail consistently, they keep getting restarted. Since the jobs keep getting restarted, the pipelines stay alive. When I checked just now, there were 8 nightly runs still running. :) Thus I'm going to cancel the still-running nightly-i386-linux-deb9-validate jobs and let the pipelines die in peace. You can still find all examples of failed jobs on the dashboard:
>> >
>> > https://grafana.gitlab.haskell.org/d/167r9v6nk/ci-spurious-failures?orgId=2&from=now-90d&to=now&refresh=5m&var-types=cannot_allocate
>> >
>> > To prevent future problems, it would be good if someone could help me look into this. Otherwise I'll just disable the job. :(
>> > _______________________________________________
>> > ghc-devs mailing list
>> > ghc-devs at haskell.org
>> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
More information about the ghc-devs
mailing list