[GHC] #15999: Stabilise nofib runtime measurements

[GHC]

#15999: Stabilise nofib runtime measurements
-------------------------------------+-------------------------------------
           Reporter:  sgraf          |             Owner:  (none)
               Type:  task           |            Status:  new
           Priority:  normal         |         Milestone:  ⊥
          Component:  NoFib          |           Version:  8.6.2
  benchmark suite                    |
           Keywords:                 |  Operating System:  Unknown/Multiple
       Architecture:                 |   Type of failure:  None/Unknown
  Unknown/Multiple                   |
          Test Case:                 |        Blocked By:
           Blocking:                 |   Related Tickets:  #5793 #9476
                                     |  #15333 #15357
Differential Rev(s):                 |         Wiki Page:
-------------------------------------+-------------------------------------
 With Phab:D4989 (cf. #15357) having hit `nofib` master, there are still
 many benchmarks that are unstable. I identified three causes for
 unstability in https://ghc.haskell.org/trac/ghc/ticket/5793#comment:38.
 With system overhead mostly out of the equation, there are still two
 related tasks left:

 1. Identify benchmarks with GC wibbles. Plan: Look at counted instructions
 while varying heap size with just one generation. A wibbling benchmark
 should have quite diverse sampled maximum residency (as opposed to a
 microbenchmark, which should have quite stable instruction count).

    Then fix these by iterating `main` 'often enough'. Maybe look at total
 bytes allocated for that, we want this to be monotonically declining as
 the initial heap size grows.
 2. Now, all benchmarks should have stable instruction count. If not, maybe
 there's another class of benchmarks I didn't identify yet in #5793. Of
 these benchmarks, there are a few, like `real/eff/CS`, that still have
 highly unstable runtimes. Fix these 'microbenchmarks' by hiding them
 behind a flag.

-- 
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/15999>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler