Slower Compilation on GHC 7.8.4 (vs. 7.6.3)

[Ozgun Ataman]

Hello,

Apologies upfront for the long post below, but I thought our experience on
this particular point might be worth sharing. I present some observations
first and then mention the particular problem around "cabal repl"
facilitated development workflow. Any ideas/feedback on the latter point
would be much appreciated and please let me know if there are other
metrics, info or output I can produce here for further
investigation/clarification.

Recently at work, we upgraded a decent size project from GHC 7.6.3 to GHC
7.8.4 and as a result observed a very significant slowdown in compile
times. Code changes were very minimal and should not be a factor. Any
mistakes below are mine, but I can fairly confidently say that there is a
visible-to-the-eye slowdown in compile times.

We made a few measurements to quantify the issue and here is the data:

   - This project has 220 direct modules (non-dependency) that are compiled
   with each "cabal build" and we always use sandboxes
   - Tests were performed on OS X, though we saw similar results on Linux
   - Single threaded (-j=1) wall-clock compile time has gone up by around
   30% for the overall project
   - With O1 and -j=N, the overall wall-clock time is approximately the
   same (around 8 min), but the CPU time spent is a staggering 7.5X higher (!)
   - With O0 and -j=N the overall wall-clock time is 40% higher, but the
   CPU time spent is a staggering 7.5X higher (!)
   - With -O0, "cabal repl" load times have gone up by a staggering 2.5X (!)
      - 7.8.4: 3 min. 35 seconds
      - 7.6.3: 1 min. 2 seconds
   - Measuring compile times for individual modules, we see that those that
   are heavily loaded with lots of types, TH-facilitated type and typeclass
   derivations and those that contain large "blobs" of values directly at the
   top level now take much longer to compile:
      - We have 10 modules that each take over 10 seconds
      - We have 3 modules that each take over 35 seconds
   - Sidenote observation: In general, parallel builds with -j appear to
   cause a very significant overhead under the "system" part of timing:
      - Example with 7.6.3, O0: cabal build  140.57s user 13.25s system
      100% cpu 2:33.70 total
      - Example with 7.8.4, O0: cabal build  507.83s user 655.43s system
      549% cpu 3:31.59 total


The main hurt here has been the infeasibility of using "cabal repl / ghci"
 fast-feedback style development (via emacs, vim, command-line, etc.),
since:

   - Unlike "cabal build", "cabal repl" re-compiles from scratch with every
   invocation ":l App.Foo.Bar", which itself is the necessary first step when
   starting to hack on a module (we can then use :r to somewhat help the
   situation)
   - The slowdown is about 2.5X as stated above: What used to take a minute
   now takes 3.5 minutes.
   - cabal repl does not seem to benefit from -j at all
   - cabal repl 7.8.4 appears to be hurt particularly by the heavier modules
   - The heavier modules are often at the top of the compile tree (types,
   derivations, etc) and are practically loaded on the critical path all the
   time

Let me know if I'm missing anything here and any/all feedback is much
appreciated!

Cheers,
Oz
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20150405/69542f0f/attachment.html>