[GHC] #9315: Weird change in allocation numbers of T9203

[GHC]

#9315: Weird change in allocation numbers of T9203
-------------------------------------+-------------------------------------
              Reporter:  nomeata     |             Owner:  simonmar
                  Type:  bug         |            Status:  new
              Priority:  normal      |         Milestone:
             Component:  Runtime     |           Version:  7.9
  System                             |          Keywords:
            Resolution:              |  Operating System:  Unknown/Multiple
Differential Revisions:              |   Type of failure:  None/Unknown
          Architecture:              |         Test Case:
  Unknown/Multiple                   |          Blocking:
            Difficulty:  Unknown     |
            Blocked By:              |
       Related Tickets:              |
-------------------------------------+-------------------------------------

Comment (by nomeata):

 I tried to bring it up on the mailing list, but nobody cared...

 > What I'd like to see instead is one of the following:
 >
 >   * Skip the test unless the libraries were built with -O2
 >   * Skip the test *if* the libraries were built with -O2
 >   * Have different results according to the optimisation levels
 >
 > All of which fix the perceived problem without introducing the
 regression in validate time.

 While the third option is the most complete, I don’t think it is
 maintainable.

 Given the aim of validate is to prevent breakage, and performance
 regressions are not breakage, I think it is ok to skip the tests there
 (even makes it faster!), and run the tests only when we are doing a
 release build.

 So how about this:
  * The expected numbers are adjusted to reflect the release settings.
  * `validate` in modes fast and normal pass `SKIP_PERF_TESTS` to the test
 suite
  * `validate` in slow mode uses the release settings (`-O2` to libraries
 and stage2), and does not `SKIP_PERF_TESTS`
  * Optionally: The test suite, when invoked manually, ignores performance
 tests if the flags are not as expected¹

 Note that the test suite already skips compiler performance tests if
 `-DDEBUG` is set, so this has precedent.

 ¹ Is that possible in a reliable and not-too-complicate way, and without
 duplicating the “expected flags”? Maybe a warning “You have a custom
 `mk/build.mk`; failures in performance tests might be irrelevant.” would
 be enough

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/9315#comment:24>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler