[GHC] #14781: Incorrect CPU core counts in virtualized environments

[GHC]

#14781: Incorrect CPU core counts in virtualized environments
-------------------------------------+-------------------------------------
        Reporter:  rtfeldman         |                Owner:  (none)
            Type:  bug               |               Status:  new
        Priority:  normal            |            Milestone:
       Component:  Core Libraries    |              Version:  8.2.2
      Resolution:                    |             Keywords:
Operating System:  Unknown/Multiple  |         Architecture:
 Type of failure:  Incorrect result  |  Unknown/Multiple
  at runtime                         |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):
       Wiki Page:                    |
-------------------------------------+-------------------------------------
Description changed by rtfeldman:

Old description:

> To reproduce this bug:
>
> 1. Write a program that prints `numCapabilities` (the # of CPU cores the
> program thinks are available for parallelization) to stdout
> 2. Run this program in a virtualized environment such as Travis CI to see
> what it prints
>
> Expected output: it prints the number of CPU cores available to the
> process, which in the case of Travis's virtualized environment is 2.
>
> Actual output: it prints the number of *physical* cores on the machine,
> which in the case of Travis's servers is 32.
>
> The real-world consequences of this bug manifest for the compiler for the
> Elm programming language, which is written in Haskell and compiled with
> GHC. By default, `elm-make` runs extremely slowly on Travis and Circle CI
> because it's trying to parallelize across 32 cores when only 2 are
> actually available to it.
>
> For a proof-of-concept for how to correctly detect the number of
> available cores (as opposed to physical), here is the source code to a
> Rust library which does so correctly:
> https://github.com/seanmonstar/num_cpus/blob/master/src/lib.rs - the
> library distinguishes between "number of CPUs" and "number of physical
> CPUs," and on Travis it correctly reports 2 for CPUs and 32 for physical.

New description:

 To reproduce this bug, use the following `main.hs`:

 {{{#!haskell
 import Control.Concurrent

 main =
   print =<< getNumCapabilities
 }}}

 Compile it with `ghc -O2 -threaded --make main.hs -o reproduce` and run
 `./reproduce` in a virtualized environment such as Travis CI.


 **Expected output:** it prints the number of CPU cores available to the
 process, which in the case of Travis's virtualized environment is 2.

 **Actual output:** it prints the number of *physical* cores on the
 machine, which in the case of Travis's servers is 32.

 == Example of Failure on Travis

 Here is an example build on Travis which demonstrates this failing:
 (scroll to the end of the console log to see the failure) https://travis-
 ci.org/rtfeldman/repro-ghc-bug/builds/340268785

 Here is the repository which was used to reproduce this failure on Travis:
 https://github.com/rtfeldman/repro-ghc-bug

 == Real-World Consequences

 The real-world consequences of this bug manifest for the compiler for the
 Elm programming language, which is written in Haskell and compiled with
 GHC. By default, `elm-make` runs extremely slowly on Travis and Circle CI
 because it's trying to parallelize across 32 cores when only 2 are
 actually available to it.

 == How to Fix

 For an example implementation which correctly detects the number of
 available cores (as opposed to physical), here is the source code to a
 Rust library which does so correctly:
 https://github.com/seanmonstar/num_cpus/blob/master/src/lib.rs - the
 library distinguishes between "number of CPUs" and "number of physical
 CPUs," and on Travis it correctly reports 2 for CPUs and 32 for physical.

--

-- 
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/14781#comment:1>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler