[GHC] #9221: (super!) linear slowdown of parallel builds on 40 core machine

[GHC]

#9221: (super!) linear slowdown of parallel builds on 40 core machine
-------------------------------------+-------------------------------------
        Reporter:  carter            |                Owner:
            Type:  bug               |               Status:  new
        Priority:  high              |            Milestone:  8.0.1
       Component:  Compiler          |              Version:  7.8.2
      Resolution:                    |             Keywords:
Operating System:  Unknown/Multiple  |         Architecture:
 Type of failure:  Compile-time      |  Unknown/Multiple
  performance bug                    |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:  #910, #8224       |  Differential Rev(s):
       Wiki Page:                    |
-------------------------------------+-------------------------------------
Description changed by bgamari:

Old description:

> im seeing slowdowns in parallel builds of a (simple!!) 6 module project
> when I build it on a 40 core server i'm using for work.  for any given
> ghc invocation with -jn, once n>10, i start to see a super linear slow
> down  as a function of n
>
> heres some basic numbers
>

> at -j1 0m2.693s
>
> at -j4  0m2.507s
>
> at -j10  0m2.763s
>
> at -j25 0m12.634s
>
> at -j30 :   0m39.154s
>
> at -j40 : 0m57.511s
>
> at -j60 : 2m21.821s
>

> these timings are another 2-4x worse if ghc is invoked indirectly via
> cabal-install / setup.hs
>
> according to the linux utility latencytop, 100% of ghc's cpu time was
> spent on user-space lock contention when I did the -j40 invocation.
>
> the timing in the -j40 case stayed the same even when ghc was also passed
> -O0 (and -fforce-recomp to ensure it did the same )
>

> a bit of experimentation makes me believe that in *ANY* cabalized project
> on a 40 core machine will exhibit this perf issue.
>
> cabal clean ; cabal configure --ghc-options="-j" ; cabal build -j1
>
> should be enough to trigger the lock contention.
>
> That said, I'll try to cook up a minimal repro that i can share the
> source for post haste.

New description:

 I'm seeing slowdowns in parallel builds of a (simple!!) 6 module project
 when I
 build it on a 40 core server that I'm using for work. For any given ghc
 invocation
 with `-jn`, once n>10, I start to see a super-linear slow down as a
 function of n,

 here are some basic numbers,

 ||=    =||= compile time =||
 || -j1  || 0m2.693s   ||
 || -j4  || 0m2.507s   ||
 || -j10 || 0m2.763s   ||
 || -j25 || 0m12.634s  ||
 || -j30 || 0m39.154s  ||
 || -j40 || 0m57.511s  ||
 || -j60 || 2m21.821s  ||

 These timings are another 2-4x worse if ghc is invoked indirectly via
 cabal-install / `Setup.hs`

 According to the linux utility latencytop, 100% of ghc's cpu time was
 spent on user-space lock contention when I did the `-j40` invocation.

 The timing in the `-j40` case stayed the same even when ghc was also
 passed `-O0` (and `-fforce-recomp` to ensure it did the same )


 A bit of experimentation makes me believe that *any* cabalized project on
 a 40 core machine will exhibit this performance issue.

 {{{
 cabal clean
 cabal configure --ghc-options="-j"
 cabal build -j1
 }}}

 should be enough to trigger the lock contention.

 That said, I'll try to cook up a minimal repro that i can share the source
 for post haste.

--

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/9221#comment:43>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler