[Git][ghc/ghc][wip/testsuite-stack-size] testsuite: Pass -kb16k -kc128k for performance tests

[Matthew Pickering (@mpickering)]

Matthew Pickering pushed to branch wip/testsuite-stack-size at Glasgow Haskell Compiler / GHC


Commits:
21163368 by Matthew Pickering at 2023-05-30T09:34:22+01:00
testsuite: Pass -kb16k -kc128k for performance tests

Setting a larger stack chunk size gives a greater protection
from stack thrashing (where the repeated overflow/underflow allocates a lot of
stack chunks which sigificantly impact allocations). This
stabilises some tests against differences cause by more things being
pushed onto the stack.

The performance tests are generally testing work done by the compiler,
using allocation as a proxy, so removing/stabilising the allocations due
to the stack gives us more stable tests which are also more sensitive
to actual changes in compiler performance.

The tests which increase are ones where we compile a lot of modules, and
for each module we spawn a thread to compile the module in. Therefore
increasing these numbers has a multiplying effect on these tests because
there are many more stacks which we can increase in size.

The most significant improvements though are cases such as T8095 which
reduce significantly in allocations (30%). This isn't a performance
improvement really but just helps stabilise the test against this
threshold set by the defaults.

Fixes #23439

-------------------------
Metric Decrease:
    InstanceMatching
    T14683
    T8095
    T9872b_defer
    T9872d
    T9961
    hie002
    T19695
    T3064
Metric Increase:
    MultiLayerModules
    T13701
    T14697
-------------------------

- - - - -


1 changed file:

- testsuite/driver/testlib.py


Changes:

=====================================
testsuite/driver/testlib.py
=====================================
@@ -1791,7 +1791,11 @@ async def simple_build(name: Union[TestName, str],
 
     stats_file = name + '.comp.stats'
     if isCompilerStatsTest():
-        extra_hc_opts += ' +RTS -V0 -t' + stats_file + ' --machine-readable -RTS'
+        # Set a bigger chunk size to reduce variation due to additional under/overflowing
+        # The tests are attempting to test how much work the compiler is doing by proxy of
+        # bytes allocated. The additional allocations caused by stack overflow can cause
+        # spurious failures if you trip over the limit (see #23439)
+        extra_hc_opts += ' +RTS -kc128k -kb16k -V0 -t' + stats_file + ' --machine-readable -RTS'
     if backpack:
         extra_hc_opts += ' -outputdir ' + name + '.out'
 



View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/commit/21163368a188eb8c692329afd744f81ecb5e2a9b

-- 
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/commit/21163368a188eb8c692329afd744f81ecb5e2a9b
You're receiving this email because of your account on gitlab.haskell.org.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-commits/attachments/20230530/8a66b49b/attachment.html>