<div dir="ltr"><div class="gmail_default" style="font-family:tahoma,sans-serif">Sorry about that!</div><div class="gmail_default" style="font-family:tahoma,sans-serif"><br></div><div class="gmail_default" style="font-family:tahoma,sans-serif">Maybe you have a giant data type with deriving(Generic)? GHC tends to behave badly on those. And yes, you seem to have a lot of type-family stuff going on! Usually we see 10k coercion sizes; you have 400k.<br></div><div class="gmail_default" style="font-family:tahoma,sans-serif"><br></div><div class="gmail_default" style="font-family:tahoma,sans-serif">Quite a lot of improvements have happened in this area, which may (or may not) help. Once you have whittled a bit, perhaps it'd be possible to test with HEAD?</div><div class="gmail_default" style="font-family:tahoma,sans-serif"><br></div><div class="gmail_default" style="font-family:tahoma,sans-serif">This was better with ... 9.6? 9.4?</div><div class="gmail_default" style="font-family:tahoma,sans-serif"><br></div><div class="gmail_default" style="font-family:tahoma,sans-serif">Simon<br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, 16 Feb 2024 at 01:36, Justin Bailey <<a href="mailto:jgbailey@gmail.com">jgbailey@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Well, after running with these flags, one of the `.dump-simpl` files<br>
is 26 GB! That's also the module it seems to hang on, so pretty sure<br>
something is going wrong there!<br>
<br>
I was seeing output indicating GHC had allocated 146GB during some of<br>
the passes ???<br>
<br>
```<br>
<br>
*** Simplifier [xxx.AirTrafficControl.Types.ATCMessage]:<br>
Result size of Simplifier iteration=1<br>
= {terms: 9,134,<br>
types: 49,937,<br>
coercions: 388,802,399,<br>
joins: 53/289}<br>
Result size of Simplifier iteration=2<br>
= {terms: 8,368,<br>
types: 46,864,<br>
coercions: 176,356,474,<br>
joins: 25/200}<br>
Result size of Simplifier<br>
= {terms: 8,363,<br>
types: 46,848,<br>
coercions: 176,356,474,<br>
joins: 25/200}<br>
!!! Simplifier [xxx.AirTrafficControl.Types.ATCMessage]: finished in<br>
294595.62 milliseconds, allocated 146497.087 megabytes<br>
```<br>
<br>
So anyways I'll continue whittling this down. This module does use a<br>
lot of higher-kinded types and fancy stuff.<br>
<br>
On Thu, Feb 15, 2024 at 3:56 PM Simon Peyton Jones<br>
<<a href="mailto:simon.peytonjones@gmail.com" target="_blank">simon.peytonjones@gmail.com</a>> wrote:<br>
><br>
> Using `-dshow-passes` is very helpful too. It shows the program size after each pass of the compiler.<br>
><br>
> Simon<br>
><br>
> On Thu, 15 Feb 2024 at 19:36, Teofil Camarasu <<a href="mailto:teofilcamarasu@gmail.com" target="_blank">teofilcamarasu@gmail.com</a>> wrote:<br>
>><br>
>> Hi Justin,<br>
>><br>
>> From your description, it sounds to me like there's something in your source code that's causing the optimiser to generate too much code, which then causes the crash because of memory exhaustion (though I might be wrong about this).<br>
>> In the past, when I've run into similar things. I've followed the following vague process to help find a minimal reproducer of the issue.<br>
>><br>
>> - pass `-ddump-simpl -ddump-timings -ddump-to-file` to GHC. (See here for docs on these flags: <a href="https://downloads.haskell.org/ghc/latest/docs/users_guide/debugging.html" rel="noreferrer" target="_blank">https://downloads.haskell.org/ghc/latest/docs/users_guide/debugging.html</a>)<br>
>> These will write some extra debugging information to either your `dist-newstyle` or `.stack-work` directory depending on whether you use cabal or stack.<br>
>> They will create for each source file a `.dump-simpl` file that will give you the compiler's intermediate output. And a `.dump-timings` file that will show timings information about how long each phase of compilation took.<br>
>><br>
>> - The first step is to hone down on the problematic module or modules. Maybe you already have a good idea from where in your build the compiler crashes.<br>
>> But if not, you can use the `.dump-timings` files and/or a tool that summarises them like <a href="https://github.com/codedownio/time-ghc-modules" rel="noreferrer" target="_blank">https://github.com/codedownio/time-ghc-modules</a>. To get a sense of where the problem lies.<br>
>><br>
>> - Once you've found your worst module, the next step is to determine what about that module is causing the issue.<br>
>> I find that often you can just try to find what top level identifiers in your `.dump-simpl` file are big. This will give a good idea of which part of your source code is to blame.<br>
>> Then I tend to try to delete everything that is irrelevant, and check again. Incrementally you get something that is smaller and smaller, and in time you tend to end up with something that is small enough to write up as a ticket.<br>
>><br>
>> I hope that helps. I've found this process to work quite well for hunting down issues where GHC's optimiser goes wrong, but it is a bit of a labour intensive process.<br>
>><br>
>> One last thing. You mention that you are on M2. If it's easily doable for you, try to reproduce on x86_64 just to make sure it's not some bug specific to M2.<br>
>><br>
>> Cheers,<br>
>> Teo<br>
>><br>
>> On Thu, Feb 15, 2024 at 7:08 PM Justin Bailey <<a href="mailto:jgbailey@gmail.com" target="_blank">jgbailey@gmail.com</a>> wrote:<br>
>>><br>
>>> Hi!<br>
>>><br>
>>> I'm trying to upgrade our (large) codebase to use 9.8.1. (I'm on an M2).<br>
>>><br>
>>> When building with -01, memory on the GHC process climbs until it<br>
>>> reaches the limit of my machine (64G) and then crashes with a<br>
>>> segfault.<br>
>>><br>
>>> With -00, that does not happen.<br>
>>><br>
>>> How would I go about diagnosing what's happening? Using RTS flags to<br>
>>> limit the heap to 32G produced the same behavior, just faster.<br>
>>><br>
>>> Strangely, `-v5` does not produce any more output in the console<br>
>>> (passed via cabal's --ghc-options). Maybe I'm doing it wrong?<br>
>>><br>
>>> Pointers to existing issues or documentation welcome! Thank you!<br>
>>><br>
>>> Justin<br>
>>> _______________________________________________<br>
>>> ghc-devs mailing list<br>
>>> <a href="mailto:ghc-devs@haskell.org" target="_blank">ghc-devs@haskell.org</a><br>
>>> <a href="http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs" rel="noreferrer" target="_blank">http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs</a><br>
>><br>
>> _______________________________________________<br>
>> ghc-devs mailing list<br>
>> <a href="mailto:ghc-devs@haskell.org" target="_blank">ghc-devs@haskell.org</a><br>
>> <a href="http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs" rel="noreferrer" target="_blank">http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs</a><br>
</blockquote></div>