Perf notes
Simon Peyton Jones
simonpj at microsoft.com
Fri Jun 26 14:16:50 UTC 2020
Also looking at gitlab CI, there are a lot of recent commits on master without completed CI runs
I thought that wasn't possible. Isn't that what CI is *for*?
And in any case, I don't think anyone will have accepted a doubling of bytes-allocated on T9803, in the last 28 commits, without lots of song and dance.
Ben do you know what is going on?
Simon
From: ghc-devs <ghc-devs-bounces at haskell.org> On Behalf Of David Eichmann
Sent: 26 June 2020 12:06
To: ghc-devs at haskell.org
Subject: Re: Perf notes
Hi Simon,
skip to the list at the bottom for TL;DR
Every time I get an email about perf notes my heart sinks a little. Hopefully there isn't a big issues here.
First of all, what commit is your branch based on? Have you rebased on a recent master?
The output you posted says "...increased from x86_64-linux-deb9 baseline @ HEAD~28". So this means it is using metrics from CI as a baseline (that's the "x86_64-linux-deb9" part), but the baseline is from 28 commits ago (that's the "HEAD~28" part). The baseline seems a bit old. Also looking at gitlab CI, there are a lot of recent commits on master without completed CI runs. So this might be a matter of waiting for CI to finish, then fetching the CI metrics again. Any way this may help:
TL;DR
1. Rebase of the latest master
2. Wait for CI to finish on a more recent commit (see https://gitlab.haskell.org/ghc/ghc/-/commits/master<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.haskell.org%2Fghc%2Fghc%2F-%2Fcommits%2Fmaster&data=02%7C01%7Csimonpj%40microsoft.com%7Cc46e95a2b82c42ce4c1608d819c10ecd%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637287664255730878&sdata=7wmJDbNNCoRb3ooWXEcxmzOqkNch4kuVNHyAQhhn0Ko%3D&reserved=0>)
3. git fetch https://gitlab.haskell.org/ghc/ghc-performance-notes.git<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.haskell.org%2Fghc%2Fghc-performance-notes.git&data=02%7C01%7Csimonpj%40microsoft.com%7Cc46e95a2b82c42ce4c1608d819c10ecd%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637287664255730878&sdata=7gipL1xRcNrHR1%2BxWZt9aS7L%2F0KzHgU31InvZk%2FRvnM%3D&reserved=0>
refs/notes/perf:refs/notes/ci/perf
4. Re run the tests
Alternatively you can generate local metrics
1. Checkout a recent commit to use as the baseline (make sure the working tree is clean)
2. Run the relevant perf tests
3. Checkout your branches HEAD commit again
4. Run the relevant tests again.
If that doesn't do it, I can have a closer look.
David E
On 6/26/20 11:29 AM, Simon Peyton Jones via ghc-devs wrote:
Despite a recent
git fetch https://gitlab.haskell.org/ghc/ghc-performance-notes.git<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.haskell.org%2Fghc%2Fghc-performance-notes.git&data=02%7C01%7Csimonpj%40microsoft.com%7Cc46e95a2b82c42ce4c1608d819c10ecd%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637287664255740875&sdata=aY3P2tJBpL3Ut8RdubHZka0hFtlnr2yhEuNSYA5OfLI%3D&reserved=0>
refs/notes/perf:refs/notes/ci/perf
I'm getting lots of perf regressions in HEAD. For example
=====> T9203(normal) 1 of 1 [0, 0, 0]
]0;T9203(normal) 1 of 1 [0, 0, 0]cd "T9203.run" && "/home/simonpj/code/HEAD-3/inplace/bin/ghc-stage2" -o T9203 T9203.hs -dcore-lint -dstg-lint -dcmm-lint -no-user-package-db -rtsopts -fno-warn-missed-specialisations -fshow-warning-groups -fdiagnostics-color=never -fno-diagnostics-show-caret -Werror=compat -dno-debug-output -O2<
cd "T9203.run" && ./T9203 +RTS -V0 -tT9203.stats --machine-readable -RTS <
runtime/bytes allocated increased from x86_64-linux-deb9 baseline @ HEAD~28:
Expected T9203 (normal) runtime/bytes allocated: 56046952.0 +/-5%
Lower bound T9203 (normal) runtime/bytes allocated: 53244604
Upper bound T9203 (normal) runtime/bytes allocated: 58849300
Actual T9203 (normal) runtime/bytes allocated: 108464536
Deviation T9203 (normal) runtime/bytes allocated: 93.5 %
*** unexpected stat test failure for T9203(normal)
Performance Metrics (test environment: local):
T9203(normal) runtime/bytes allocated 108464536.000
(baseline @ HEAD~28) 56046952.000 [increased, 93.5%]
What am I doing wrong?
Simon
_______________________________________________
ghc-devs mailing list
ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs<https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.haskell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-devs&data=02%7C01%7Csimonpj%40microsoft.com%7Cc46e95a2b82c42ce4c1608d819c10ecd%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637287664255740875&sdata=7qu5Ji4KN61LG8RP1X7%2B3GG2kq%2FfSUbnthjV33sYubs%3D&reserved=0>
--
David Eichmann, Haskell Consultant
Well-Typed LLP, http://www.well-typed.com<https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.well-typed.com%2F&data=02%7C01%7Csimonpj%40microsoft.com%7Cc46e95a2b82c42ce4c1608d819c10ecd%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637287664255750861&sdata=VL3fotTvZCGb0wVbrlzmIGHkGEYyqPAlN%2BWd5RRXEN0%3D&reserved=0>
Registered in England & Wales, OC335890
118 Wymering Mansions, Wymering Road, London W9 2NF, England
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20200626/104fa14e/attachment.html>
More information about the ghc-devs
mailing list