Darwin CI Status

[Matthew Pickering]

Thanks Moritz for that update.

The latest is that currently darwin CI is disabled and the merge train
is unblocked (*choo choo*).

I am testing Moritz's patches to speed-up CI and will merge them in
shortly to get darwin coverage back.

Cheers,

Matt

On Wed, May 19, 2021 at 9:46 AM Moritz Angermann
<moritz.angermann at gmail.com> wrote:
>
> Matt has access to the M1 builder in my closet now. The darwin performance issue
> is mainly there since BigSur, and (afaik) primarily due to the amount of DYLD_LIBRARY_PATH's
> we pass to GHC invocations. The system linker spends the majority of the time in the
> kernel stat'ing and getelements (or some similar directory) call for each and every possible
> path.
>
> Switching to hadrian will cut down the time from ~5hs to ~2hs. At some point we had make
> builds <90min by just killing all DYLD_LIBRARY_PATH logic we ever had, but that broke
> bindists.
>
> The CI has time values attached and some summary at the end right now, which highlights
> time spent in the system and in user mode. This is up to 80% sys, 20% user, and went to
> something like 20% sys, 80% user after nuking all DYLD_LIBRARY_PATH's, with hadrian it's
> closer to ~25% sys, 75% user.
>
> Of note, this is mostly due to time spent during the *test-suite*, not the actual build. For the
> actual build make and hadrian are comparable, though I've seen hadrian to oddly have a
> much higher variance in how long it takes to *build* ghc, whereas the make build was more
> consistent.
>
> The test-suite quite notoriously calls GHC *a lot of times*, which makes any linker issue due
> to DYLD_LIBRARY_PATH (and similar lookups) much worse.
>
> If we would finally split building and testing, we'd see this more clearly I believe. Maybe this
> is motivation enough for someone to come forward to break build/test into two CI steps?
>
> Cheers,
>  Moritz
>
> On Wed, May 19, 2021 at 4:14 PM Matthew Pickering <matthewtpickering at gmail.com> wrote:
>>
>> Hi all,
>>
>> The darwin pipelines are gumming up the merge pipeline as they are
>> taking over 4 hours to complete on average.
>>
>> I am going to disable them -
>> https://gitlab.haskell.org/ghc/ghc/-/merge_requests/5785
>>
>> Please can someone give me access to one of the M1 builders so I can
>> debug why the tests are taking so long. Once I have fixed the issue
>> then I will enable the pipelines.
>>
>> Cheers,
>>
>> Matt