Memory usage exploding for complex pattern matching

[Ben Gamari]

"Victor Miraldo (UU)" <v.cacciarimiraldo at uu.nl> writes:

> Hello Ben,
>
>
>>> We came up with a "minimal" example that at least is self contained and
>>> triggers the problem; `runghc Minimal.hs` should finish with about 5GB of RAM.
>>>
>> Hmm, with 8.2 I'm seeing a peak memory usage of 3.5 GB, with that peak
>> occurring during renaming, not desugaring (when exhaustiveness checking
>> happens). Are you certain that the exhaustiveness checker is the
>> responsible party?
>
> Not at all! In fact, I ran some comparisons with "-ddump-phases" and
> plotted those.
> I'm attaching the graphs here. From base to "added E" we added 16 more
> pattern matches,
> then each "added" phase is an additional 16 pattern matches. The
> simplifier stands out there.
>
> I couldn't run a bigger case as I had hit the ceiling of my machine.
>
I tried 8.4 and indeed the renamer/typechecker takes dramatically less
memory (yay!), leaving desugar as the largest allocator (allocating
around 3GB) while compiling the minimal example. Furthermore, disabling
the pattern match checker (via -fmax-pmcheck-iterations=0) dramatically
reduces desugarer allocations (to around 60MByte). It looks like you are right.

Unfortunately, it's not clear what can be done to work around this short of
disabling the match checker for the entire module, as suggested by
mpickering. Ideally we would have some notion of local warning flags
(e.g. #602), allowing us to disable this (rather expensive) warning for
generated code. Sadly, we aren't there yet. Perhaps you would like to
propose such a mechanism?

Cheers,

- Ben

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20180329/e8043069/attachment.sig>