[GHC] #9221: (super!) linear slowdown of parallel builds on 40 core machine

[GHC]

#9221: (super!) linear slowdown of parallel builds on 40 core machine
-------------------------------------+-------------------------------------
        Reporter:  carter            |                Owner:
            Type:  bug               |               Status:  new
        Priority:  high              |            Milestone:  8.0.1
       Component:  Compiler          |              Version:  7.8.2
      Resolution:                    |             Keywords:
Operating System:  Unknown/Multiple  |         Architecture:
 Type of failure:  Compile-time      |  Unknown/Multiple
  performance bug                    |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:  #910, #8224       |  Differential Rev(s):
       Wiki Page:                    |
-------------------------------------+-------------------------------------

Comment (by rrnewton):

 If blackholing *is* the culprit then I wonder how localized the
 strictification will need to be...

 Ezyang's multi-process suggestion seems appealing.  But if it's a
 `forkProcess` approach, is it possible to load most of the relevant state
 into memory before forking a child process per module?  That could avoid
 or reduce communication through files or shared pages.

 The granularity of per-module compiles is big, so maybe processes would be
 ok.  Also, there's the nice side effect that the GCs of child processes
 are disentangled, removing that particular scaling bottleneck.

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/9221#comment:42>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler