[GHC] #9221: (super!) linear slowdown of parallel builds on 40 core machine

[GHC]

#9221: (super!) linear slowdown of parallel builds on 40 core machine
-------------------------------------+-------------------------------------
        Reporter:  carter            |                Owner:
            Type:  bug               |               Status:  new
        Priority:  normal            |            Milestone:  8.2.1
       Component:  Compiler          |              Version:  7.8.2
      Resolution:                    |             Keywords:
Operating System:  Unknown/Multiple  |         Architecture:
 Type of failure:  Compile-time      |  Unknown/Multiple
  performance bug                    |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:  #910, #8224       |  Differential Rev(s):
       Wiki Page:                    |
-------------------------------------+-------------------------------------

Comment (by slyfox):

 Replying to [comment:56 simonmar]:
 > Yes, perhaps we should default to -qb0 when -A is larger than some
 threshold.
 >
 > The mutator parallelism looks not too bad (3.24 out of 4).  Was there
 enough parallelism in the program you were compiling to do better than
 that?

 I assumed there is as it's ~100 independent haskell modules,
 but some of them are disproportionally large.

 Got to 24-core VM to run the same benchmark. At first it manages to peak
 24 HEC utilisation
 but at the tail it clearly has not enough parallelism:

 -qb1: http://code.haskell.org/~slyfox/T9221/ghc-
 stage2.eventlog.N24.j24.png

 -qb0: http://code.haskell.org/~slyfox/T9221/ghc-
 stage2.eventlog.N24.j24.A256M.qb0.png

 > The `MVar`s are probably just the compilation manager: it fires up a
 thread for every module, and they wait on the results of compiling the
 modules they depend on.

 Yes, that's more likely. It would be cool to have labelMVar similar to
 existing labelThread
 to see IDs/names of blocker MVars.

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/9221#comment:57>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler