[GHC] #15374: Testsuite: compile_timeout_multiplier is fragile

[GHC]

#15374: Testsuite: compile_timeout_multiplier is fragile
-------------------------------------+-------------------------------------
           Reporter:  goldfire       |             Owner:  (none)
               Type:  bug            |            Status:  new
           Priority:  normal         |         Milestone:  8.6.1
          Component:  Compiler       |           Version:  8.4.3
           Keywords:                 |  Operating System:  Unknown/Multiple
       Architecture:                 |   Type of failure:  None/Unknown
  Unknown/Multiple                   |
          Test Case:                 |        Blocked By:
           Blocking:                 |   Related Tickets:
Differential Rev(s):                 |         Wiki Page:
-------------------------------------+-------------------------------------
 I'm taking a sweep through the testsuite, trying to eliminate failures in
 a `DEBUG` compiler. But two tests are stymieing me:
 `pmcheck/should_compile/T11{276,374}`. Both of these use the
 `compile_timout_multiplier` option in their `all.T` entries. This option
 means that, instead of the normal 300s allotted to the test, these tests
 get only 3s. In my setup, with a `DEBUG` (read: unoptimized) stage2
 compiler, these tests fail when I'm running the testsuite in parallel.
 Then, when I run the tests individually, they succeed.

 This poses two problems:

 1. I can only see the failure when my machine is stressed, and it's
 disconcerting to have a failure appear and disappear depending on how I
 look.

 2. There is (sometimes) a failure in `DEBUG` mode.

 What is the goal of `compile_timeout_multiplier`? If we're really testing
 performance (which #11276 and #11374 indicate), then these tests should be
 in `perf/compiler` and be "stat" tests, where some wibbling in the numbers
 is expected (and where `DEBUG` compilers are exempted). I would imagine
 that `compile_timeout_multiplier` should only ever get values greater than
 1, when a non-performance test is so expensive to run that it takes more
 than 300s.

 As a local solution here, I think these two tests should move to
 `perf/compiler`. Other pattern-match tests also have
 `compile_timeout_multiplier`, too, and I can move these as well. Any
 objections?

-- 
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/15374>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler