[Git][ghc/ghc][wip/backport-MR769] 10 commits: testsuite: Introduce fragile modifier

Ben Gamari gitlab at gitlab.haskell.org
Wed Jun 12 12:40:52 UTC 2019



Ben Gamari pushed to branch wip/backport-MR769 at Glasgow Haskell Compiler / GHC


Commits:
175d49e6 by Ben Gamari at 2019-06-12T00:07:25Z
testsuite: Introduce fragile modifier

Now since we have been a bit more stringent in testsuite cleanliness we
have been marking a lot of tests as fragile using the `skip` modifier.
However, this unfortunately means that we lose the association with the
ticket number documenting the fragility.

Here we introduce `fragile` and `fragile_for` to retain this
information.

(cherry picked from commit 4ca271d1880a6f4c5f49869de7f1920a2073adb6)

- - - - -
18ffc386 by Ben Gamari at 2019-06-12T00:07:25Z
testsuite: Mark heapprof001 as fragile on i386

(cherry picked from commit 910185a3eb5fd2148e42d39f6374ab03d098b682)

- - - - -
a9a34082 by Ben Gamari at 2019-06-12T00:07:25Z
testsuite: Mark heapprof001 as fragile on all platforms

See #15382.

(cherry picked from commit 23fc615679072a6fa433460a92f597af2ae388b2)

- - - - -
9c385294 by Ben Gamari at 2019-06-12T00:07:25Z
base: Mark CPUTime001 as fragile

As noted in #16224, CPUTime001 has been quite problematic, reporting
non-monotonic timestamps in CI. Unfortunately I've been unable to
reproduce this locally.

(cherry picked from commit 1a3420cabdcf6d7d90c154681230f1150604c097)

- - - - -
c993dee9 by Ben Gamari at 2019-06-12T00:07:25Z
testsuite: Mark T13167 as fragile in threaded2

As noted in #16536.

(cherry picked from commit b351004702c1a595bcedfa3ffeb4f816d5fd8503)

- - - - -
5993703c by Ben Gamari at 2019-06-12T00:07:25Z
testsuite: Fix fragile_for test modifier

(cherry picked from commit 658199cce0aabeed77f3bbbbde6abc0c5c3cc83d)

- - - - -
5e6f261a by Ben Gamari at 2019-06-12T00:07:25Z
testsuite: Mark threadstatus-T9333 as fragile in ghci way

As noted in #16555.

(cherry picked from commit 64b1684da09ddb3dc480bd0370adc7b002657a39)

- - - - -
c4b501b1 by Takenobu Tani at 2019-06-12T12:39:41Z
Add `-haddock` option under ci condition to fix #16415

In order to use the `:doc` command in ghci, it is necessary
to compile for core libraries with `-haddock` option.

Especially, the `-haddock` option is essential for release building.

Note:
  * The `-haddock` option may affect compile time and binary size.
  * But hadrian has already set `-haddock` as the default.
  * This patch affects the make-based building.

This patch has been split from !532.

(cherry picked from commit 33e37d0619a9d1d0b8088a109f7eeb4c6fd21027)

- - - - -
b51f9ecd by Takenobu Tani at 2019-06-12T12:39:41Z
Add `-haddock` to perf.mk rather than prepare-system.sh

To cover ci conditions from ghc8.6 to 8.9, I add `-haddock` option
to `mk/flavours/perf.mk` rather than `.circleci/prepare-system.sh`.

Because in windows condition of ghc-8.9, `mk/flavours/*` is included
after `prepare-system.sh`.

In addition, in linux condition of ghc-8.6, `mk/flavors/perf.mk` is used.

(cherry picked from commit 43a39c3c2195d5b4400efc845a54f153184b1d7f)

- - - - -
ca7173a9 by Takenobu Tani at 2019-06-12T12:39:41Z
Add `-haddock` to prepare-system.sh and .gitlab-ci.yml

To cover ci conditions from ghc8.6 to 8.9, I add `-haddock` option
to `.circleci/prepare-system.sh` and .gitlab-ci.yml.
after including `mk/flavours/*`.

(cherry picked from commit c4f94320a7048a7f263d8d952d4e12cc0227cf72)

- - - - -


6 changed files:

- .circleci/prepare-system.sh
- .gitlab-ci.yml
- libraries/base/tests/all.T
- testsuite/driver/testlib.py
- testsuite/tests/concurrent/should_run/all.T
- testsuite/tests/profiling/should_run/all.T


Changes:

=====================================
.circleci/prepare-system.sh
=====================================
@@ -30,6 +30,7 @@ BuildFlavour=$BUILD_FLAVOUR
 ifneq "\$(BuildFlavour)" ""
 include mk/flavours/\$(BuildFlavour).mk
 endif
+GhcLibHcOpts+=-haddock
 EOF
 
 case "$(uname)" in


=====================================
.gitlab-ci.yml
=====================================
@@ -563,6 +563,7 @@ nightly-i386-windows-hadrian:
       python boot
       bash -c './configure --enable-tarballs-autodownload GHC=`pwd`/toolchain/bin/ghc HAPPY=`pwd`/toolchain/bin/happy ALEX=`pwd`/toolchain/bin/alex $CONFIGURE_ARGS'
     - bash -c "echo include mk/flavours/${BUILD_FLAVOUR}.mk > mk/build.mk"
+    - bash -c "echo 'GhcLibHcOpts+=-haddock' >> mk/build.mk"
     - bash -c "PATH=`pwd`/toolchain/bin:$PATH make -j`mk/detect-cpu-count.sh`"
     - bash -c "PATH=`pwd`/toolchain/bin:$PATH make binary-dist TAR_COMP_OPTS=-1"
     - bash -c 'make V=0 test THREADS=`mk/detect-cpu-count.sh` JUNIT_FILE=../../junit.xml'


=====================================
libraries/base/tests/all.T
=====================================
@@ -20,7 +20,7 @@ test('tempfiles', normal, compile_and_run, [''])
 test('fixed', normal, compile_and_run, [''])
 test('quotOverflow', normal, compile_and_run, [''])
 test('assert', exit_code(1), compile_and_run, ['-fno-ignore-asserts'])
-test('CPUTime001', normal, compile_and_run, [''])
+test('CPUTime001', fragile(16224), compile_and_run, [''])
 test('readLitChar',   normal, compile_and_run, [''])
 test('unicode001',
      when(platform('i386-unknown-openbsd'), expect_fail),
@@ -231,5 +231,5 @@ test('T3474',
 test('T14425', normal, compile_and_run, [''])
 test('T10412', normal, compile_and_run, [''])
 test('T13896', normal, compile_and_run, [''])
-test('T13167', normal, compile_and_run, [''])
+test('T13167', fragile_for(16536, ['threaded2']), compile_and_run, [''])
 test('T15349', [exit_code(1), expect_broken_for(15349, 'ghci')], compile_and_run, [''])


=====================================
testsuite/driver/testlib.py
=====================================
@@ -225,11 +225,35 @@ def _expect_pass(way):
 
 # -----
 
+def fragile( bug ):
+    """
+    Indicates that the test should be skipped due to fragility documented in
+    the given ticket.
+    """
+    def helper( name, opts, bug=bug ):
+        record_broken(name, opts, bug)
+        opts.skip = True
+
+    return helper
+
+def fragile_for( bug, ways ):
+    """
+    Indicates that the test should be skipped due to fragility in the given
+    test ways as documented in the given ticket.
+    """
+    def helper( name, opts, bug=bug, ways=ways ):
+        record_broken(name, opts, bug)
+        opts.omit_ways += ways
+
+    return helper
+
+# -----
+
 def omit_ways( ways ):
     return lambda name, opts, w=ways: _omit_ways( name, opts, w )
 
 def _omit_ways( name, opts, ways ):
-    opts.omit_ways = ways
+    opts.omit_ways += ways
 
 # -----
 


=====================================
testsuite/tests/concurrent/should_run/all.T
=====================================
@@ -89,7 +89,7 @@ test('T7970', normal, compile_and_run, [''])
 test('AtomicPrimops', normal, compile_and_run, [''])
 
 # test uses 2 threads and yield, scheduling can vary with threaded2
-test('threadstatus-9333', [omit_ways(['threaded2'])], compile_and_run, [''])
+test('threadstatus-9333', [fragile_for(16555, ['ghci']), omit_ways(['threaded2'])], compile_and_run, [''])
 
 test('T9379', normal, compile_and_run, [''])
 


=====================================
testsuite/tests/profiling/should_run/all.T
=====================================
@@ -24,7 +24,7 @@ expect_broken_for_10037 = expect_broken_for(
 
 test('heapprof001',
      [when(have_profiling(), extra_ways(extra_prof_ways)),
-      when(arch('i386'), expect_broken_for(15382, ['prof_hc_hb'])),
+      fragile(15382),
       extra_run_opts('7')],
      compile_and_run, [''])
 



View it on GitLab: https://gitlab.haskell.org/ghc/ghc/compare/b867c2c379330d212b3c223e4b5cfc49defc0e61...ca7173a9902184d4f76957765a352715d75c2670

-- 
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/compare/b867c2c379330d212b3c223e4b5cfc49defc0e61...ca7173a9902184d4f76957765a352715d75c2670
You're receiving this email because of your account on gitlab.haskell.org.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-commits/attachments/20190612/c8b0d6bd/attachment-0001.html>


More information about the ghc-commits mailing list