[GHC] #15614: Test suite should show reason for skipping a test

[GHC]

#15614: Test suite should show reason for skipping a test
-------------------------------------+-------------------------------------
           Reporter:  osa1           |             Owner:  (none)
               Type:  task           |            Status:  new
           Priority:  normal         |         Milestone:
          Component:  Test Suite     |           Version:  8.5
           Keywords:                 |  Operating System:  Unknown/Multiple
       Architecture:                 |   Type of failure:  None/Unknown
  Unknown/Multiple                   |
          Test Case:                 |        Blocked By:
           Blocking:                 |   Related Tickets:
Differential Rev(s):                 |         Wiki Page:
-------------------------------------+-------------------------------------
 I want to add a GHCi test, but it's a bit tricky to run the test so I use
 a
 make rule and use `run_command`, like this:

 {{{
 test('UnsafeReenterGhci',
      [only_ways(['ghci']), exit_code(1), extra_files(['UnsafeReenter.hs',
 'UnsafeReenterC.c'])],
      run_command,
      ['$MAKE -s --no-print-directory UnsafeReenterGhci'])
 }}}

 The important part is `run_command` and `only_ways(['ghci'])`. The problem
 is
 that it's impossible to run this test, because by default a `run_command`
 test
 can only be run in normal way (see test_common_work() in testlib.py), but
 there's no way to know this without reading the source code.

 Having better documentation might help, but I think the test driver should
 print why a test is skipped (maybe only in verbose mode -- if such a mode
 exists). In our case it should print something like "Test is only run in
 'normal' way" (because `run_command` tests only run in 'normal' way).

 Correct version of this test is:

 {{{
 test('UnsafeReenterGhci',
      [extra_ways(['ghci']), only_ways(['ghci']), exit_code(1),
 extra_files(['UnsafeReenter.hs', 'UnsafeReenterC.c'])],
      run_command,
      ['$MAKE -s --no-print-directory UnsafeReenterGhci'])
 }}}

 The `extra_ways` makes the test driver consider running the test in `ghci`
 way,
 `only_ways` is to avoid running it in `normal` way (the default and only
 way
 for `run_command`).

-- 
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/15614>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler