[Git][ghc/ghc][master] 2 commits: testsuite/perf: Report better error message on malformed note

[Marge Bot (@marge-bot)]

Marge Bot pushed to branch master at Glasgow Haskell Compiler / GHC


Commits:
f28b5992 by Ben Gamari at 2024-10-02T22:23:47-04:00
testsuite/perf: Report better error message on malformed note

Previously a malformed perf note resulted in very poor errors.
Here we slight improve this situation.

- - - - -
51377508 by Ben Gamari at 2024-10-02T22:23:47-04:00
testsuite: Handle division-by-zero more gracefully

Previously we would fail with an ZeroDivisionError.

Fixes #25321

- - - - -


2 changed files:

- testsuite/driver/perf_notes.py
- testsuite/driver/runtests.py


Changes:

=====================================
testsuite/driver/perf_notes.py
=====================================
@@ -165,6 +165,9 @@ class RelativeMetricAcceptanceWindow(MetricAcceptanceWindow):
 
 def parse_perf_stat(stat_str: str) -> PerfStat:
     field_vals = stat_str.strip('\t').split('\t')
+    if len(field_vals) != 5:
+        raise ValueError(f'Invalid stat line: {field_vals}')
+
     stat = PerfStat(*field_vals) # type: ignore
     if stat.test_env.startswith('"') and stat.test_env.endswith('"'):
         # Due to a bug, in historical data sometimes the test_env
@@ -183,11 +186,14 @@ def get_perf_stats(commit: Union[GitRef, GitHash]=GitRef('HEAD'),
     except subprocess.CalledProcessError:
         return []
 
-    return \
-        [ parse_perf_stat(stat_str)
-          for stat_str in log.strip('\n').split('\n')
-          if stat_str != ''
-        ]
+    try:
+        return \
+            [ parse_perf_stat(stat_str)
+              for stat_str in log.strip('\n').split('\n')
+              if stat_str != ''
+            ]
+    except ValueError as e:
+        raise ValueError(f'Invalid stat line for commit {commit}')
 
 # Check if a str is in a 40 character git commit hash.
 _commit_hash_re = re.compile('[0-9a-f]' * 40)
@@ -659,7 +665,10 @@ def check_stats_change(actual: PerfStat,
         display('    Upper bound ' + full_name + ' ' + actual.metric + ':', upperBound, '')
         display('    Actual      ' + full_name + ' ' + actual.metric + ':', actual.value, '')
         if actual.value != expected_val:
-            actual_dev = round(((float(actual.value) * 100)/ int(expected_val)) - 100, 1)
+            if expected_val == 0:
+                actual_dev = 100.0
+            else:
+                actual_dev = round(((float(actual.value) * 100)/ int(expected_val)) - 100, 1)
             display('    Deviation   ' + full_name + ' ' + actual.metric + ':', actual_dev, '%')
 
     return (change, result)


=====================================
testsuite/driver/runtests.py
=====================================
@@ -404,7 +404,12 @@ def tabulate_metrics(metrics: List[PerfMetric]) -> None:
             return ""
         val0 = x.baseline.perfStat.value
         val1 = x.stat.value
-        return "{:+2.1f}%".format(100 * (val1 - val0) / val0)
+        if val0 == 0 and val1 == 0:
+            return "0.0%"
+        elif val0 == 0:
+            return "NaN%"
+        else:
+            return "{:+2.1f}%".format(100 * (val1 - val0) / val0)
     dataRows = [row((
         "{}({})".format(x.stat.test, x.stat.way),
         shorten_metric_name(x.stat.metric),
@@ -425,6 +430,7 @@ def tabulate_metrics(metrics: List[PerfMetric]) -> None:
         x.stat.value / x.baseline.perfStat.value
         for x in metrics
         if x.baseline is not None
+        if x.baseline.perfStat.value != 0
     ]
     minimum = 0.0
     maximum = 0.0



View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/115a30e9142b4481de3ba735396e9d0417d46445...513775082b89deae3f83896031caf0e89a7ed333

-- 
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/115a30e9142b4481de3ba735396e9d0417d46445...513775082b89deae3f83896031caf0e89a7ed333
You're receiving this email because of your account on gitlab.haskell.org.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-commits/attachments/20241002/a1fdea4c/attachment-0001.html>