[Git][ghc/ghc][wip/T17987] Deleted 1 commit: testsuite: Don't consider stat measurements from broken tests"

[Ben Gamari]

Ben Gamari pushed to branch wip/T17987 at Glasgow Haskell Compiler / GHC


WARNING: The push did not contain any new commits, but force pushed to delete the commits and changes below.


Deleted commits:
1de44aa1 by Ben Gamari at 2020-03-30T14:07:32-04:00
testsuite: Don't consider stat measurements from broken tests"

Previously we would add statistics from tests marked as broken to the stats
output. This broke in #17987 since the test was considered to be "broken"
solely on the basis of its allocations. In later testsuite runs the "broken"
allocations metric was then considered to be the baseline and the test started
unexpectedly passing.

We now ignore metrics that arise from tests marked as broken. Of course, this
required that we distinguish between "broken" and merely "expected to fail". I
took this opportunity to do a bit of refactoring in our representation of test
outcomes.

- - - - -


1 changed file:

- testsuite/driver/testlib.py


Changes:

=====================================
testsuite/driver/testlib.py
=====================================
@@ -1407,6 +1407,10 @@ def check_stats(name: TestName,
                 stats_file: Path,
                 range_fields: Dict[MetricName, MetricOracles]
                 ) -> PassFail:
+    if getTestOpts().expect == ExpectedOutcome.BROKEN:
+        print('Skipping performance metrics test on broken test {}'.format(name))
+        return passed()
+
     head_commit = Perf.commit_hash(GitRef('HEAD')) if Perf.inside_git_repo() else None
     if head_commit is None:
         return passed()



View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/commit/1de44aa1c09ee35b7775354f380f8c9054b1220a

-- 
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/commit/1de44aa1c09ee35b7775354f380f8c9054b1220a
You're receiving this email because of your account on gitlab.haskell.org.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-commits/attachments/20200330/f663075b/attachment.html>