[Git][ghc/ghc][master] testsuite: Improve output from tests which have failing pre_cmd

[Marge Bot (@marge-bot)]

Marge Bot pushed to branch master at Glasgow Haskell Compiler / GHC


Commits:
8f6c576b by Matthew Pickering at 2022-11-14T16:48:45-05:00
testsuite: Improve output from tests which have failing pre_cmd

There are two changes:

* If a pre_cmd fails, then don't attempt to run the test.
* If a pre_cmd fails, then print the stdout and stderr from running that
  command (which hopefully has a nice error message).

For example:

```
=====> 1 of 1 [0, 0, 0]
*** framework failure for test-defaulting-plugin(normal) pre_cmd failed: 2
** pre_cmd was "$MAKE -s --no-print-directory -C defaulting-plugin package.test-defaulting-plugin TOP={top}".
stdout:
stderr:
DefaultLifted.hs:19:13: error: [GHC-76037]
    Not in scope: type constructor or class ‘Typ’
    Suggested fix:
      Perhaps use one of these:
        ‘Type’ (imported from GHC.Tc.Utils.TcType),
        data constructor ‘Type’ (imported from GHC.Plugins)
   |
19 | instance Eq Typ where
   |             ^^^
make: *** [Makefile:17: package.test-defaulting-plugin] Error 1

Performance Metrics (test environment: local):
```

Fixes #22329

- - - - -


1 changed file:

- testsuite/driver/testlib.py


Changes:

=====================================
testsuite/driver/testlib.py
=====================================
@@ -1247,14 +1247,23 @@ def do_test(name: TestName,
             dst_makefile.write_text(makefile, encoding='UTF-8')
 
     if opts.pre_cmd:
+        stdout_path = in_testdir(name, 'pre_cmd_stdout')
+        stderr_path = in_testdir(name, 'pre_cmd_stderr')
         exit_code = runCmd('cd "{0}" && {1}'.format(opts.testdir, override_options(opts.pre_cmd)),
-                           stderr = subprocess.STDOUT,
+                           stdout = stdout_path,
+                           stderr = stderr_path,
                            print_output = config.verbose >= 3)
 
         # If user used expect_broken then don't record failures of pre_cmd
         if exit_code != 0 and opts.expect not in ['fail']:
             framework_fail(name, way, 'pre_cmd failed: {0}'.format(exit_code))
             if_verbose(1, '** pre_cmd was "{0}".'.format(override_options(opts.pre_cmd)))
+            stderr_contents = stderr_path.read_text(encoding='UTF-8', errors='replace')
+            stdout_contents = stdout_path.read_text(encoding='UTF-8', errors='replace')
+            if_verbose(1, 'stdout: {0}'.format(stdout_contents))
+            if_verbose(1, 'stderr: {0}'.format(stderr_contents))
+            # Don't continue and try to run the test if the pre_cmd fails.
+            return
 
     result = func(*[name,way] + args)
 



View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/commit/8f6c576b0b9b82acf23c51ae8cb3c6e5bde61ab4

-- 
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/commit/8f6c576b0b9b82acf23c51ae8cb3c6e5bde61ab4
You're receiving this email because of your account on gitlab.haskell.org.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-commits/attachments/20221114/cd5035e5/attachment-0001.html>