[commit: ghc] master: testsuite: Move echoing commands in make invocations to VERBOSE=5 (6421c6f)

[git at git.haskell.org]

Repository : ssh://git@git.haskell.org/ghc

On branch  : master
Link       : http://ghc.haskell.org/trac/ghc/changeset/6421c6f2bb56bcdf2415e8556180821eedd77d2d/ghc

>---------------------------------------------------------------

commit 6421c6f2bb56bcdf2415e8556180821eedd77d2d
Author: Reid Barton <rwbarton at gmail.com>
Date:   Thu Mar 2 16:39:16 2017 -0500

    testsuite: Move echoing commands in make invocations to VERBOSE=5
    
    D2894 added a new verbosity level VERBOSE=4 to strip -s/--silent
    flags from make invocations in test commands. This will probably
    cause the test to fail of course, but is useful for seeing what
    a test that's already failing is doing.
    
    However there was already an undocumented meaning of VERBOSE=4,
    added in commit cfeededf, that causes the results of performance
    tests to be printed unconditionally (even when they are within the
    expected range). nomeata's ghc builder uses these figures to
    collect historical data on performance test figures. The new
    meaning of VERBOSE=4 added in D2894 means that any test that uses
    make now fails on the builder.
    
    This commit moves the new behavior of D2894 to the level VERBOSE=5
    so that nomeata's ghc builder again produces useful results on
    failing tests. It also adds documentation for both settings.
    
    Test Plan: did some manual testing
    
    Reviewers: austin, bgamari, Phyx, nomeata
    
    Reviewed By: bgamari, Phyx
    
    Subscribers: nomeata, thomie, Phyx
    
    Differential Revision: https://phabricator.haskell.org/D3141


>---------------------------------------------------------------

6421c6f2bb56bcdf2415e8556180821eedd77d2d
 testsuite/README.md          | 4 +++-
 testsuite/driver/runtests.py | 4 ++--
 testsuite/driver/testlib.py  | 2 +-
 3 files changed, 6 insertions(+), 4 deletions(-)

diff --git a/testsuite/README.md b/testsuite/README.md
index 31193d6..5ab6daf 100644
--- a/testsuite/README.md
+++ b/testsuite/README.md
@@ -18,7 +18,9 @@ Commands to run testsuite:
  * Skip performance tests: `make SKIP_PERF_TESTS=YES`
  * Set verbosity: `make VERBOSE=n`
    where n=0: No per-test ouput, n=1: Only failures,
-         n=2: Progress output, n=3: Include commands called (default)
+         n=2: Progress output, n=3: Include commands called (default),
+         n=4: Include perf test results unconditionally,
+         n=5: Echo commands in subsidiary make invocations
  * Pass in extra GHC options: `make EXTRA_HC_OPTS=-fvectorize`
 
 You can also change directory to a specific test folder to run that
diff --git a/testsuite/driver/runtests.py b/testsuite/driver/runtests.py
index df85fa1..7e4f375 100644
--- a/testsuite/driver/runtests.py
+++ b/testsuite/driver/runtests.py
@@ -111,8 +111,8 @@ for opt,arg in opts:
         config.skip_perf_tests = True
 
     if opt == '--verbose':
-        if arg not in ["0","1","2","3","4"]:
-            sys.stderr.write("ERROR: requested verbosity %s not supported, use 0,1,2,3 or 4" % arg)
+        if arg not in ["0","1","2","3","4","5"]:
+            sys.stderr.write("ERROR: requested verbosity %s not supported, use 0,1,2,3,4 or 5" % arg)
             sys.exit(1)
         config.verbose = int(arg)
 
diff --git a/testsuite/driver/testlib.py b/testsuite/driver/testlib.py
index 44d1ccb..7dedb33 100644
--- a/testsuite/driver/testlib.py
+++ b/testsuite/driver/testlib.py
@@ -872,7 +872,7 @@ def do_test(name, way, func, args, files):
 # if found and instead have the testsuite decide on what to do
 # with the output.
 def override_options(pre_cmd):
-    if config.verbose >= 4 and bool(re.match('\$make', pre_cmd, re.I)):
+    if config.verbose >= 5 and bool(re.match('\$make', pre_cmd, re.I)):
         return pre_cmd.replace('-s'      , '') \
                       .replace('--silent', '') \
                       .replace('--quiet' , '')