[Git][ghc/ghc][wip/req_smp] testsuite: req_smp --> req_target_smp, req_ghc_smp

doyougnu (@doyougnu) gitlab at gitlab.haskell.org
Mon Jan 16 15:06:10 UTC 2023



doyougnu pushed to branch wip/req_smp at Glasgow Haskell Compiler / GHC


Commits:
244e5f1e by Sylvain Henry at 2023-01-16T10:05:51-05:00
testsuite: req_smp --> req_target_smp, req_ghc_smp

See #22630 and !9552

This commit:
 - splits req_smp into req_target_smp and req_ghc_smp
 - changes the testsuite driver to calculate req_ghc_smp
 - changes a handful of tests to use req_target_smp instead of req_smp
 - changes a handful of tests to use req_host_smp when needed

The problem:
 - the problem this solves is the ambiguity surrounding req_smp
 - on master req_smp was used to express the constraint that the program
 being compiled supports smp _and_ that the host RTS (i.e., the RTS used
 to compile the program) supported smp. Normally that is fine, but in
 cross compilation this is not always the case as was discovered in #22630.

The solution:
 - Differentiate the two constraints:
   - use req_target_smp to say the RTS the compiled program is linked
   with (and the platform) supports smp
   - use req_host_smp to say the RTS the host is linked with supports smp

WIP: fix req_smp (target vs ghc)

add flag to separate bootstrapper

split req_smp -> req_target_smp and req_ghc_smp

update tests smp flags

cleanup and add some docstrings

only set ghc_with_smp to bootstrapper on S1 or CC

Only set ghc_with_smp to bootstrapperWithSMP of when testing stage 1
and cross compiling

test the RTS in config/ghc not hadrian

re-add ghc_with_smp

fix and align req names

fix T11760 to use req_host_smp

test the rts directly, avoid python 3.5 limitation

test the compiler in a try block

align out of tree and in tree withSMP flags

mark failing tests as host req smp

testsuite: req_host_smp --> req_ghc_smp

Fix ghc vs host, fix ghc_with_smp leftover

- - - - -


16 changed files:

- hadrian/src/Settings/Builders/RunTest.hs
- libraries/base/tests/all.T
- testsuite/config/ghc
- testsuite/driver/testglobals.py
- testsuite/driver/testlib.py
- testsuite/mk/test.mk
- testsuite/tests/codeGen/should_run/all.T
- testsuite/tests/concurrent/T13615/all.T
- testsuite/tests/concurrent/should_run/all.T
- testsuite/tests/driver/T14075/all.T
- testsuite/tests/driver/T20030/test1/all.T
- testsuite/tests/driver/j-space/all.T
- testsuite/tests/driver/t22391/all.T
- testsuite/tests/ffi/should_run/all.T
- testsuite/tests/rts/all.T
- testsuite/tests/typecheck/should_fail/all.T


Changes:

=====================================
hadrian/src/Settings/Builders/RunTest.hs
=====================================
@@ -68,8 +68,8 @@ data TestCompilerArgs = TestCompilerArgs{
  ,   withInterpreter   :: Bool
  ,   unregisterised    :: Bool
  ,   tables_next_to_code :: Bool
- ,   withSMP           :: Bool
- ,   debugAssertions   :: Bool
+ ,   targetWithSMP       :: Bool  -- does the target support SMP
+ ,   debugAssertions     :: Bool
       -- ^ Whether the compiler has debug assertions enabled,
       -- corresponding to the -DDEBUG option.
  ,   profiled          :: Bool
@@ -100,7 +100,7 @@ inTreeCompilerArgs stg = do
     withInterpreter     <- ghcWithInterpreter
     unregisterised      <- flag GhcUnregisterised
     tables_next_to_code <- flag TablesNextToCode
-    withSMP             <- targetSupportsSMP
+    targetWithSMP       <- targetSupportsSMP
     debugAssertions     <- ($ succStage stg) . ghcDebugAssertions <$> flavour
     profiled            <- ghcProfiled        <$> flavour <*> pure stg
 
@@ -145,8 +145,8 @@ outOfTreeCompilerArgs = do
     withNativeCodeGen   <- getBooleanSetting TestGhcWithNativeCodeGen
     withInterpreter     <- getBooleanSetting TestGhcWithInterpreter
     unregisterised      <- getBooleanSetting TestGhcUnregisterised
-    tables_next_to_code <- getBooleanSetting TestGhcTablesNextToCode
-    withSMP             <- getBooleanSetting TestGhcWithSMP
+    tables_next_to_code <- getBooleanSetting TestGhcUnregisterised
+    targetWithSMP       <- targetSupportsSMP
     debugAssertions     <- getBooleanSetting TestGhcDebugged
 
     os          <- getTestSetting TestHostOS
@@ -202,7 +202,7 @@ runTestBuilderArgs = builder Testsuite ? do
     bignumBackend <- getBignumBackend
     bignumCheck   <- getBignumCheck
 
-    keepFiles           <- expr (testKeepFiles <$> userSetting defaultTestArgs)
+    keepFiles <- expr (testKeepFiles <$> userSetting defaultTestArgs)
 
     accept <- expr (testAccept <$> userSetting defaultTestArgs)
     (acceptPlatform, acceptOS) <- expr . liftIO $
@@ -262,8 +262,7 @@ runTestBuilderArgs = builder Testsuite ? do
             , arg "-e", arg $ asBool "ghc_with_dynamic_rts="  (hasDynamicRts)
             , arg "-e", arg $ asBool "ghc_with_threaded_rts=" (hasThreadedRts)
             , arg "-e", arg $ asBool "config.have_fast_bignum=" (bignumBackend /= "native" && not bignumCheck)
-            , arg "-e", arg $ asBool "ghc_with_smp=" withSMP
-
+            , arg "-e", arg $ asBool "target_with_smp=" targetWithSMP
             , arg "-e", arg $ "config.ghc_dynamic=" ++ show hasDynamic
             , arg "-e", arg $ "config.leading_underscore=" ++ show leadingUnderscore
 


=====================================
libraries/base/tests/all.T
=====================================
@@ -240,7 +240,7 @@ test('T11555', normal, compile_and_run, [''])
 test('T12494', normal, compile_and_run, [''])
 test('T12852', [when(opsys('mingw32'), skip), js_broken(22374)], compile_and_run, [''])
 test('lazySTexamples', normal, compile_and_run, [''])
-test('T11760', req_smp, compile_and_run, ['-threaded -with-rtsopts=-N2'])
+test('T11760', [req_ghc_smp, req_target_smp], compile_and_run, ['-threaded -with-rtsopts=-N2'])
 test('T12874', normal, compile_and_run, [''])
 test('T13191',
         [ collect_stats('bytes allocated', 5)


=====================================
testsuite/config/ghc
=====================================
@@ -43,10 +43,12 @@ if ghc_with_native_codegen:
 if config.have_interp:
     config.run_ways.append('ghci')
 
+# we read the 'Support SMP' setting from the ghcconfig file. This dictates
+# whether the target supports smp
 if ghc_with_threaded_rts:
     config.run_ways.append('threaded1')
-    if ghc_with_smp:
-        config.have_smp = True
+    if target_with_smp:
+        config.target_has_smp = True
         config.run_ways.append('threaded2')
         if config.speed == 0:
             config.run_ways.append('nonmoving_thr')
@@ -210,31 +212,52 @@ def get_compiler_info():
     # See Note [Replacing backward slashes in config.libdir].
     config.libdir = config.libdir.replace('\\', '/')
 
-    def test_compile(flags) -> bool:
+    def test_compile(flags):
         """
-        Check whether GHC can compile in the given way.
-        This is used as a proxy to determine, e.g., whether
-        profiled libraries were built.
+        Check whether GHC can compile in the given way. This is used as a
+        proxy to determine, e.g., whether profiled libraries were built, or
+        whether the host RTS supports smp.
         """
         import tempfile
         import textwrap
+
+        res = False
+
         with tempfile.TemporaryDirectory() as d:
             src = Path(d) / 'test.hs'
             src.write_text(textwrap.dedent('''
                 module Main where
                 main = putStrLn "Hello World!"
             '''))
-            p = subprocess.run(
+            try:
+                p = subprocess.run(
                     '{} -v0 {} -o test '.format(config.compiler, src) + ' '.join(flags),
                     shell=True,
                     cwd=d,
-                    stderr=None if config.verbose >= 3 else subprocess.DEVNULL)
-            res = p.returncode
-            return res == 0
+                    stderr=None if config.verbose >= 3 else subprocess.DEVNULL
+                    )
+
+            except Exception as err:
+                print("Exception thrown in testsuite/config/ghc.get_compiler_info: %s", err)
+
+            else:
+                res = p.returncode == 0
+
+        return res
+
+    def compiler_supports_way(flags):
+        return test_compile(flags)
+
+    # Test the Host RTS to determine if it supports SMP. For cross compilers the
+    # Host /= Target, so we cannot determine from the ghcconfig file if the host
+    # itself supports smp. To support smp the host must be linked with an RTS
+    # built with 'defined(THREADED_RTS) && !defined(NO_SMP)'. Thus we directly
+    # query the RTS the host is linked with.
+    config.ghc_has_smp    = test_compile(["+RTS", "-N"])
 
-    config.have_vanilla = test_compile([])
-    config.have_dynamic = test_compile(['-dynamic'])
-    config.have_profiling = test_compile(['-prof'])
+    config.have_vanilla   = compiler_supports_way([])
+    config.have_dynamic   = compiler_supports_way(['-dynamic'])
+    config.have_profiling = compiler_supports_way(['-prof'])
 
     if config.have_profiling:
         config.compile_ways.append('profasm')


=====================================
testsuite/driver/testglobals.py
=====================================
@@ -147,8 +147,11 @@ class TestConfig:
         # Is the compiler dynamically linked?
         self.ghc_dynamic = False
 
-        # Do we have SMP support?
-        self.have_smp = False
+        # Does the host RTS have SMP support?
+        self.ghc_has_smp = True
+
+        # Does the target have SMP support?
+        self.target_has_smp = True
 
         # Is gdb available?
         self.have_gdb = False


=====================================
testsuite/driver/testlib.py
=====================================
@@ -296,8 +296,20 @@ def req_th( name, opts ):
     if ghc_dynamic():
         return _omit_ways(name, opts, ['profasm', 'profthreaded'])
 
-def req_smp( name, opts ):
-    if not config.have_smp:
+def req_ghc_smp( name, opts ):
+    """
+    Mark a test as requiring GHC to be linked with an RTS that supports smp.
+    """
+    if not config.ghc_has_smp:
+        opts.skip = True
+
+def req_target_smp( name, opts ):
+    """
+    Mark a test as requiring smp when run on the target. If the target does
+    not support smp, then mark the test as an expected fail. Use this when the
+    test needs to run with smp support.
+    """
+    if not config.target_has_smp:
         opts.expect = 'fail'
 
 def ignore_stdout(name, opts):


=====================================
testsuite/mk/test.mk
=====================================
@@ -164,9 +164,9 @@ CABAL_PLUGIN_BUILD = --enable-library-vanilla --disable-shared
 endif
 
 ifeq "$(GhcWithSMP)" "YES"
-RUNTEST_OPTS += -e ghc_with_smp=True
+RUNTEST_OPTS += -e target_with_smp=True
 else
-RUNTEST_OPTS += -e ghc_with_smp=False
+RUNTEST_OPTS += -e target_with_smp=False
 endif
 
 ifeq "$(GhcWithRtsLinker)" "YES"


=====================================
testsuite/tests/codeGen/should_run/all.T
=====================================
@@ -156,7 +156,7 @@ test('T10246', normal, compile_and_run, [''])
 test('T9533', normal, compile_and_run, [''])
 test('T9533b', normal, compile_and_run, [''])
 test('T9533c', normal, compile_and_run, [''])
-test('T10414', [only_ways(['threaded2']), extra_ways(['threaded2']), req_smp],
+test('T10414', [only_ways(['threaded2']), extra_ways(['threaded2']), req_target_smp, req_ghc_smp],
      compile_and_run, ['-feager-blackholing'])
 test('T10521', normal, compile_and_run, [''])
 test('T10521b', normal, compile_and_run, [''])


=====================================
testsuite/tests/concurrent/T13615/all.T
=====================================
@@ -1,7 +1,7 @@
 test('T13615',
      [when(fast(), skip),
       when(unregisterised(), skip),
-      req_smp, # needs -N support
+      req_target_smp, # needs -N support
       only_ways(threaded_ways),
       extra_files(['Parallel.hs', 'Memo.hs']),
       # Decrease stack chunk size and lots of capabilities to increase failure


=====================================
testsuite/tests/concurrent/should_run/all.T
=====================================
@@ -240,7 +240,9 @@ test('setnumcapabilities001',
      [ only_ways(['threaded1','threaded2', 'nonmoving_thr', 'profthreaded']),
        extra_run_opts('8 12 2000'),
        when(have_thread_sanitizer(), expect_broken(18808)),
-       req_smp ],
+       req_target_smp,
+       req_ghc_smp
+     ],
      compile_and_run, [''])
 
 test('T21651',
@@ -248,7 +250,9 @@ test('T21651',
        when(opsys('mingw32'),skip), # uses POSIX pipes
        when(opsys('darwin'),extra_run_opts('8 12 2000 100')),
        unless(opsys('darwin'),extra_run_opts('8 12 2000 200')), # darwin runners complain of too many open files
-       req_smp ],
+       req_target_smp,
+       req_ghc_smp
+     ],
      compile_and_run, [''])
 
 test('hs_try_putmvar001',


=====================================
testsuite/tests/driver/T14075/all.T
=====================================
@@ -1,6 +1,6 @@
 test('T14075',
      [ extra_files(['F.hs', 'F.hs-boot', 'O.hs', 'V.hs', 'V.hs-boot'])
-     , req_smp # uses ghc --make -j2
+     , req_ghc_smp # uses ghc --make -j2
      , js_broken(22261)
      ],
      makefile_test, [])


=====================================
testsuite/tests/driver/T20030/test1/all.T
=====================================
@@ -9,6 +9,7 @@ test('T20030_test1j',
      [ extra_files([ 'A.hs-boot' , 'A.hs' , 'B.hs' , 'C.hs-boot' , 'C.hs'
                    , 'D.hs' , 'E.hs-boot' , 'E.hs' , 'F.hs' , 'G.hs' , 'H.hs'
                    , 'I.hs', 'J.hs-boot', 'J.hs', 'K.hs' ])
-     , req_smp
+     , req_target_smp
+     , req_ghc_smp
      ],
      multimod_compile, ['I.hs K.hs', '-v1 -j'])


=====================================
testsuite/tests/driver/j-space/all.T
=====================================
@@ -1 +1 @@
-test('jspace', [extra_files(['genJspace']), req_smp], makefile_test, ['jspace'])
+test('jspace', [extra_files(['genJspace']), req_target_smp, req_ghc_smp], makefile_test, ['jspace'])


=====================================
testsuite/tests/driver/t22391/all.T
=====================================
@@ -1,5 +1,5 @@
 test('t22391', [extra_files(['src'])],
      multimod_compile, ['Lib', '-v1 -Wall -fhide-source-paths -isrc -fdefer-diagnostics'])
 
-test('t22391j', [req_smp, extra_files(['src'])],
+test('t22391j', [req_target_smp, req_ghc_smp, extra_files(['src'])],
      multimod_compile, ['Lib', '-v1 -Wall -fhide-source-paths -isrc -fdefer-diagnostics -j2'])


=====================================
testsuite/tests/ffi/should_run/all.T
=====================================
@@ -235,7 +235,7 @@ test('T17471', [omit_ways(['ghci']), req_c], compile_and_run,
      ['T17471_c.c -optc-D -optcFOO'])
 
 test('IncallAffinity',
-     [req_smp, only_ways(['threaded1', 'threaded2']),
+     [req_target_smp, req_ghc_smp, only_ways(['threaded1', 'threaded2']),
       # Unregisterised build doesn't support
       when(unregisterised(), skip),
       req_c],


=====================================
testsuite/tests/rts/all.T
=====================================
@@ -287,7 +287,7 @@ test('stablename001', expect_fail_for(['hpc']), compile_and_run, [''])
 
 test('T7815', [ multi_cpu_race,
                 extra_run_opts('50000 +RTS -N2 -RTS'),
-                req_smp,
+                req_target_smp, req_ghc_smp,
                 only_ways(['threaded1', 'threaded2']) ], compile_and_run, [''] )
 
 # ignore_stderr because it contains a unique:
@@ -306,10 +306,10 @@ test('T7919', [ when(fast(), skip)
 
 test('T8035', normal, compile_and_run, [''])
 
-test('T8209', [ req_smp, only_ways(threaded_ways), ignore_stdout ],
+test('T8209', [ req_target_smp, req_ghc_smp, only_ways(threaded_ways), ignore_stdout ],
               compile_and_run, [''])
 
-test('T8242', [ req_smp, only_ways(threaded_ways), ignore_stdout ],
+test('T8242', [ req_target_smp, req_ghc_smp, only_ways(threaded_ways), ignore_stdout ],
               compile_and_run, [''])
 
 test('T8124', [ only_ways(threaded_ways), omit_ways(['ghci']),
@@ -330,7 +330,8 @@ test('T9078', only_ways(['threaded1']), compile_and_run, [''])
 
 test('T10017', [ when(opsys('mingw32'), skip)
                , when(unregisterised(), skip)
-               , req_smp
+               , req_target_smp
+               , req_ghc_smp
                , only_ways(threaded_ways), extra_run_opts('+RTS -N2 -RTS') ], compile_and_run, [''])
 
 test('T11108', normal, compile_and_run, [''])
@@ -403,7 +404,7 @@ test('T11788', [ when(ghc_dynamic(), skip)
                , req_interp
                ], makefile_test, ['T11788'])
 
-test('T10296a', [req_smp], makefile_test, ['T10296a'])
+test('T10296a', [req_ghc_smp], makefile_test, ['T10296a'])
 
 test('T10296b', [only_ways(['threaded2'])], compile_and_run, [''])
 
@@ -457,7 +458,8 @@ test('alloccounter1', js_broken(22261), compile_and_run,
 
 test('nursery-chunks1',
   [ extra_run_opts('4 100 +RTS -n32k -A1m -RTS')
-  , req_smp
+  , req_ghc_smp
+  , req_target_smp
   , only_ways(['threaded1','threaded2'])
   ],
   compile_and_run,


=====================================
testsuite/tests/typecheck/should_fail/all.T
=====================================
@@ -412,7 +412,7 @@ test('T11990a', normal, compile_fail, [''])
 test('T11990b', normal, compile_fail, [''])
 test('T12035', [], multimod_compile_fail, ['T12035', '-v0'])
 test('T12035j', [ extra_files(['T12035.hs', 'T12035a.hs', 'T12035.hs-boot'])
-                , req_smp
+                , req_ghc_smp
                 , js_broken(22261)
                 ], multimod_compile_fail, ['T12035', '-j2 -v0'])
 test('T12045b', normal, compile_fail, [''])



View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/commit/244e5f1e10c86811d949f9fdc0cb05800feb09de

-- 
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/commit/244e5f1e10c86811d949f9fdc0cb05800feb09de
You're receiving this email because of your account on gitlab.haskell.org.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-commits/attachments/20230116/6e0b2e11/attachment-0001.html>


More information about the ghc-commits mailing list