[Haskell-cafe] Speeding up compilation of a happy generated parser

[Nathan Bloomfield]

Hello all! I'm using Alex/Happy to generate a parser for PHP. This is my
first go at using a parser generator instead of parser combinators and it's
been going pretty smoothly so far, but I've hit a snag.

Compiling the happy-generated parser is excruciatingly slow -- 20+ minutes
with -O0 -- which messes up the typically very fast feedback loop
haskell+ghci allows. Running the code is great; my quickcheck test suite is
very fast. But compiling and loading into ghci are so slow that iterating
on the project further is not practical until I can figure out how to speed
it up.

Are there any folklore techniques for dealing with this problem, or any
resources about how to profile/debug slow compilation?

I'm on macos using Stack with lts-13.21, which includes happy-1.19.10.

I tried the ideas in this known issue:
https://github.com/simonmar/happy/issues/109. I also tried downgrading to
lts-6.35 (the most recent with GHC 7.*), all with similar results.

I imagine some of this slowdown is inevitable because of the grammar's
size; it has ~275 nonterminals and the generated `Parser.hs` module is
3.8MB. Usually for a problem like this I'd break the module up into smaller
pieces that won't change as often, but in this case it's an opaque monolith
generated by happy. I also see that the generated code doesn't have type
signatures on all top level functions.

Here's the relevant output from GHC with -v2 enabled:

```
*** Parser [PHP.Parse.Parser]:
!!! Parser [PHP.Parse.Parser]: finished in 1980.24 milliseconds, allocated
3155.733 megabytes
*** Renamer/typechecker [PHP.Parse.Parser]:
!!! Renamer/typechecker [PHP.Parse.Parser]: finished in 243923.86
milliseconds, allocated 178189.005 megabytes
*** Desugar [PHP.Parse.Parser]:
Result size of Desugar (before optimization)
  = {terms: 1,346,664,
     types: 672,082,252,
     coercions: 789,
     joins: 0/9,651}
Result size of Desugar (after optimization)
  = {terms: 834,353,
     types: 285,053,589,
     coercions: 246,053,
     joins: 235/735}
!!! Desugar [PHP.Parse.Parser]: finished in 103983.68 milliseconds,
allocated 17597.213 megabytes
*** Simplifier [PHP.Parse.Parser]:
Result size of Simplifier iteration=1
  = {terms: 1,059,012,
     types: 69,942,882,
     coercions: 1,934,739,
     joins: 235/1,935}
Result size of Simplifier iteration=2
  = {terms: 815,760,
     types: 64,805,160,
     coercions: 1,368,873,
     joins: 233/1,929}
Result size of Simplifier
  = {terms: 776,400,
     types: 64,474,214,
     coercions: 1,297,320,
     joins: 233/1,929}
!!! Simplifier [PHP.Parse.Parser]: finished in 197842.50 milliseconds,
allocated 116983.797 megabytes
*** CoreTidy [PHP.Parse.Parser]:
Result size of Tidy Core
  = {terms: 776,400,
     types: 64,474,214,
     coercions: 1,297,320,
     joins: 233/1,929}
!!! CoreTidy [PHP.Parse.Parser]: finished in 97776.92 milliseconds,
allocated 11254.968 megabytes
*** CorePrep [PHP.Parse.Parser]:
Result size of CorePrep
  = {terms: 795,714,
     types: 66,237,082,
     coercions: 1,297,320,
     joins: 233/4,234}
!!! CorePrep [PHP.Parse.Parser]: finished in 7911.78 milliseconds,
allocated 2564.067 megabytes
*** Stg2Stg:
*** CodeGen [PHP.Parse.Parser]:
!!! CodeGen [PHP.Parse.Parser]: finished in 7048.73 milliseconds, allocated
6407.563 megabytes
*** Assembler:
*** CorePrep [PHP.Parse.Parser]:
Result size of CorePrep
  = {terms: 795,714,
     types: 66,237,082,
     coercions: 1,297,320,
     joins: 233/4,234}
!!! CorePrep [PHP.Parse.Parser]: finished in 3069.67 milliseconds,
allocated 2564.018 megabytes
*** Stg2Stg:
*** CodeGen [PHP.Parse.Parser]:
!!! CodeGen [PHP.Parse.Parser]: finished in 5970.62 milliseconds, allocated
6634.422 megabytes
*** Assembler:
*** Deleting temp files:
Warning: deleting non-existent
/var/folders/td/sxyy9wl919740vddr49g8nth0000gn/T/ghc1272_0/ghc_70.s
Warning: deleting non-existent
/var/folders/td/sxyy9wl919740vddr49g8nth0000gn/T/ghc1272_0/ghc_72.c
Warning: deleting non-existent
/var/folders/td/sxyy9wl919740vddr49g8nth0000gn/T/ghc1272_0/ghc_74.c
```

-- 
Nathan Bloomfield
Systems Analyst, Team Neutron
Automattic, Inc.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/haskell-cafe/attachments/20190515/f9e83967/attachment.html>