Memory usage exploding for complex pattern matching

[Simon Peyton Jones]

|  In order to try to hack my way through and get my library to compile,
|  I tried progressively adding more and more pattern synonyms to try to
|  avoid exhaustiveness checking,yet, something really surprising
|  happened. Big examples have type errors where small ones don't!

That is strange.  Sounds as if you want a new Trac report, just for that.

Simon

|  -----Original Message-----
|  From: Victor Miraldo (UU) <v.cacciarimiraldo at uu.nl>
|  Sent: 07 April 2018 12:02
|  To: Ben Gamari <ben at smart-cactus.org>
|  Cc: Simon Peyton Jones <simonpj at microsoft.com>; ghc-devs at haskell.org
|  Subject: Re: Memory usage exploding for complex pattern matching
|  
|  Hello all,
|  
|  Following up on this, I have created a trac ticket
|  https://ghc.haskell.org/trac/ghc/ticket/14987#no2 as suggested a
|  couple days ago.
|  
|  In order to try to hack my way through and get my library to compile,
|  I tried progressively adding more and more pattern synonyms to try to
|  avoid exhaustiveness checking,yet, something really surprising
|  happened. Big examples have type errors where small ones don't!
|  
|  I tried documenting the process in a repository:
|  https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithu
|  b.com%2FVictorCMiraldo%2Fghc-14987-repro-
|  pipeline&data=02%7C01%7Csimonpj%40microsoft.com%7C59fdacad7f94412d665e
|  08d59c7717f3%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C1%7C636586957847
|  493826&sdata=bnuBOL%2FELkmJahidX%2BmWPSDDf%2BuKlJkBAJyFVyY%2Bel8%3D&re
|  served=0
|  
|  Unfortunately, however, I couldn't get the self contained repros in
|  the repository above to throw type errors. Hence, I'm attaching my
|  full code. Compiling the file src/Generics/MRSOP/Examples/GoAST.hs
|  gives type errors, even though it is being generated by the same
|  template haskell code as the other files inside Examples, all of which
|  compile just fine. To trigger the error, just run `stack build`.
|  
|  It really is a shame that the combination of pattern synonyms that
|  shows the best memory performance crashes completely on bigger
|  examples.
|  
|  Any suggestions are appreciated, thank you very much!
|  
|  Have a great weekend!
|  Cheers,
|  Victor
|  
|  
|  
|  On Fri, Mar 30, 2018 at 5:23 PM, Ben Gamari <ben at smart-cactus.org>
|  wrote:
|  > "Victor Miraldo (UU)" <v.cacciarimiraldo at uu.nl> writes:
|  >
|  >> Hello,
|  >>
|  >>>>>> I have just tried compiling my code with 8.4.2 and using
|  >>>>>> -fmax-pmcheck-iterations=0, I gave GHC 12GB of ram and it still
|  >>>>>> ran out (through `ulimit -v$((1024*1024*12))`).
|  >>>>>>
|  >>>>> Hmmm, I'm a bit confused. Why are our results so different? How
|  >>>>> precisely are you invoking GHC?
|  >>>>
|  >>>> Here I meant my whole code, not just the repro. I could have been
|  more clear.
|  >>>> Nevertheless, I'm calling it through stack:
|  >>>>
|  >>> I'll admit that I am a bit lost; Minimal.hs compiles for me with a
|  >>> maximum residency of ~3.5 GBytes with both -O1 and the PM check
|  >>> enabled using GHC 8.4.1. Is this not the repro you are referring
|  to?
|  >>
|  >> I get the same behavior as you for Minimal.hs.
|  >>
|  >> The "my code" above referred to the whole library that I'm
|  developping.
|  >> In fact, the Minimal.hs file contains a distilled version of that
|  >> library with a template haskell splice that we are trying to use in
|  >> one of our fully fledged examples.
|  >>
|  > Okay, I just wanted to be certain we were indeed seeing the same
|  > behavior. Indeed 3.5 GB is quite a lot of memory for a 700 LoC
|  > program, even one with the deep matches seen in Minimal.hs.
|  >
|  > Cheers,
|  >
|  > - Ben
|  >