[GHC] #11501: Building nofib/fibon returns permission denied

[GHC]

#11501: Building nofib/fibon returns permission denied
-------------------------------------+-------------------------------------
        Reporter:  rem               |                Owner:
            Type:  bug               |               Status:  new
        Priority:  normal            |            Milestone:
       Component:  NoFib benchmark   |              Version:  7.10.3
  suite                              |
      Resolution:                    |             Keywords:
Operating System:  Linux             |         Architecture:  x86_64
                                     |  (amd64)
 Type of failure:  None/Unknown      |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):
       Wiki Page:                    |
-------------------------------------+-------------------------------------

Comment (by bgamari):

 I have also used `nofib` a great deal to locate past regressions in
 compiler performance regressions.

 Your observations are generally correct. I think it would be great to
 enable more tests by default. However, it's not at all clear to me that we
 necessarily always want to use `--make`. The current use of one-shot mode
 makes it significantly easier to narrow down what code in particular GHC
 has regressed on. Moving to `--make` may better reflect `Cabal`'s usage,
 but I think it would make the common case of locating compiler performance
 regressions a bit harder.

 In general I think there is certainly room for a broader performance
 testsuite consisting of a sampling of packages from Hackage. However, I'm
 not sure that `nofib` is that place. Keeping `nofib` a set of standalone
 tests without dependencies makes it a significantly smaller maintenance
 burden that the Cabal-centric approach that you describe. We as a
 community (myself included!) have had a poor record of following through
 with maintaining our performance infrastructure. Consequently I think
 there is value in keeping at least one testsuite which "just works" with
 minimal maintenance.

 I gave a talk at HIW this year describing a very rough tool that I
 developed while tracking down performance regressions in nofib using the
 performance build bot which nomeata stood up to server
 [[http://perf.haskell.org/|gipeda]]. It essentially provides the nice(?)
 graphs which you describe. Sadly, I've not had a chance to put it up in a
 publicly accessible location. In the meantime it can be found at
 http://home.smart-cactus.org/~ben/ghc-perf/. The source is available
 [[https://github.com/bgamari/ghc-perf-import/|here]].

 The general workflow is,
  1. navigate to http://home.smart-cactus.org/~ben/ghc-perf/
  2. stand in awe of my poor front-end design aesthetic
  3. select a test environment on the right-hand pane (e.g. my own build
 bot or nomeata's; the latter has better coverage).
  4. select a set of tests to plot from the left-hand pane. The tests list
 can be filtered by substring match using the text field at the top; I
 usually filter by `compile-allocs`. Loading the results may take a while,
 sadly.
  5. select a change threshold percentage such that commits which exhibit
 large changes in the selected tests show up in the commit list at the
 bottom

 This is how I typically find a starting point to dive into performance
 work. There are several types of tests captured,

  * Characterizing GHC's performance
    * `compile-allocs`: These are the compiler allocations for compiling
 each module of each test
    * `compile-time`: These are the compiler runtimes for compiling each
 module of each test
  * Compiled code characteristics
    * `binary-size`: This is the size of the produced executable
    * `module-size`: The size of each module's object file
  * Characterizing compiled code performance
    * `allocs`: This is the runtime allocations of each test
    * `gc-time`: The time spent in garbage collection while running the
 test
    * `elapsed-time`: The wall-clock time of the test run
    * `mut-elapsed-time`: The wall-clock time spent in the mutator
    * `mut-time`: CPU time spent in the mutator
    * `run-time`: The overall CPU time spent in the test

 Note that in general runtimes are very unstable. I find it most helpful to
 first identify potential regressions by looking at allocations, then look
 at runtime to see which allocations changes actually move real runtime.

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/11501#comment:17>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler