[GHC] #9221: (super!) linear slowdown of parallel builds on 40 core machine

[GHC]

#9221: (super!) linear slowdown of parallel builds on 40 core machine
-------------------------+-------------------------------------------------
       Reporter:         |             Owner:
  carter                 |            Status:  new
           Type:  bug    |         Milestone:
       Priority:  high   |           Version:  7.8.2
      Component:         |  Operating System:  Unknown/Multiple
  Compiler               |   Type of failure:  Compile-time performance bug
       Keywords:         |         Test Case:
   Architecture:         |          Blocking:
  Unknown/Multiple       |
     Difficulty:         |
  Unknown                |
     Blocked By:         |
Related Tickets:         |
-------------------------+-------------------------------------------------
 im seeing slowdowns in parallel builds of a (simple!!) 6 module project
 when I build it on a 40 core server i'm using for work.  for any given
 ghc invocation with -jn, once n>10, i start to see a super linear slow
 down  as a function of n

 heres some basic numbers


 at -j1 0m2.693s

 at -j4  0m2.507s

 at -j10  0m2.763s

 at -j25 0m12.634s

 at -j30 :   0m39.154s

 at -j40 : 0m57.511s

 at -j60 : 2m21.821s


 these timings are another 2-4x worse if ghc is invoked indirectly via
 cabal-install / setup.hs

 according to the linux utility latencytop, 100% of ghc's cpu time was
 spent on user-space lock contention when I did the -j40 invocation.

 the timing in the -j40 case stayed the same even when ghc was also passed
 -O0 (and -fforce-recomp to ensure it did the same )


 a bit of experimentation makes me believe that in *ANY* cabalized project
 on a 40 core machine will exhibit this perf issue.

 cabal clean ; cabal configure --ghc-options="-j" ; cabal build -j1

 should be enough to trigger the lock contention.

 That said, I'll try to cook up a minimal repro that i can share the source
 for post haste.

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/9221>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler