<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin-top:6.0pt;
margin-right:0cm;
margin-bottom:6.0pt;
margin-left:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif;
mso-fareast-language:EN-US;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
pre
{mso-style-priority:99;
mso-style-link:"HTML Preformatted Char";
margin:0cm;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";
mso-fareast-language:EN-GB;}
p.Code, li.Code, div.Code
{mso-style-name:Code;
margin-top:0cm;
margin-right:0cm;
margin-bottom:0cm;
margin-left:22.7pt;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";
mso-fareast-language:EN-US;
font-weight:bold;}
span.HTMLPreformattedChar
{mso-style-name:"HTML Preformatted Char";
mso-style-priority:99;
mso-style-link:"HTML Preformatted";
font-family:Consolas;
mso-fareast-language:EN-US;}
span.EmailStyle23
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-GB" link="#0563C1" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-US">Also looking at gitlab CI, there are a lot of recent commits on master without completed CI runs<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">I thought that wasn’t possible. Isn’t that what CI is *<b>for</b>*?<o:p></o:p></span></p>
<p class="MsoNormal">And in any case, I don’t think anyone will have accepted a doubling of bytes-allocated on T9803, in the last 28 commits, without lots of song and dance.<o:p></o:p></p>
<p class="MsoNormal">Ben do you know what is going on?<o:p></o:p></p>
<p class="MsoNormal">Simon<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0cm 0cm 0cm 4.0pt">
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal" style="margin:0cm;margin-bottom:.0001pt"><b><span lang="EN-US" style="mso-fareast-language:EN-GB">From:</span></b><span lang="EN-US" style="mso-fareast-language:EN-GB"> ghc-devs <ghc-devs-bounces@haskell.org>
<b>On Behalf Of </b>David Eichmann<br>
<b>Sent:</b> 26 June 2020 12:06<br>
<b>To:</b> ghc-devs@haskell.org<br>
<b>Subject:</b> Re: Perf notes<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p>Hi Simon,<span style="mso-fareast-language:EN-GB"><o:p></o:p></span></p>
<p>skip to the list at the bottom for TL;DR<o:p></o:p></p>
<p>Every time I get an email about perf notes my heart sinks a little. Hopefully there isn't a big issues here.<o:p></o:p></p>
<p>First of all, what commit is your branch based on? Have you rebased on a recent master?<o:p></o:p></p>
<p>The output you posted says "<span lang="EN-US">...increased from x86_64-linux-deb9 baseline @ HEAD~28". So this means it is using metrics from CI as a baseline (that's the "x86_64-linux-deb9" part), but the baseline is from 28 commits ago (that's the "HEAD~28"
part). The baseline seems a bit old. Also looking at gitlab CI, there are a lot of recent commits on master without completed CI runs. So this might be a matter of waiting for CI to finish, then fetching the CI metrics again. Any way this may help:</span><o:p></o:p></p>
<p><o:p> </o:p></p>
<p><span lang="EN-US">TL;DR</span><o:p></o:p></p>
<p><o:p> </o:p></p>
<p><span lang="EN-US">1. Rebase of the latest master</span><o:p></o:p></p>
<p><span lang="EN-US">2. Wait for CI to finish on a more recent commit (see <a href="https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.haskell.org%2Fghc%2Fghc%2F-%2Fcommits%2Fmaster&data=02%7C01%7Csimonpj%40microsoft.com%7Cc46e95a2b82c42ce4c1608d819c10ecd%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637287664255730878&sdata=7wmJDbNNCoRb3ooWXEcxmzOqkNch4kuVNHyAQhhn0Ko%3D&reserved=0">
https://gitlab.haskell.org/ghc/ghc/-/commits/master</a>)</span><o:p></o:p></p>
<p><span lang="EN-US">3. git fetch <a href="https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.haskell.org%2Fghc%2Fghc-performance-notes.git&data=02%7C01%7Csimonpj%40microsoft.com%7Cc46e95a2b82c42ce4c1608d819c10ecd%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637287664255730878&sdata=7gipL1xRcNrHR1%2BxWZt9aS7L%2F0KzHgU31InvZk%2FRvnM%3D&reserved=0">
https://gitlab.haskell.org/ghc/ghc-performance-notes.git</a></span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US">refs/notes/perf:refs/notes/ci/perf</span><span lang="EN-US">
</span><o:p></o:p></p>
<p><span lang="EN-US">4. Re run the tests</span><o:p></o:p></p>
<p><o:p> </o:p></p>
<p><span lang="EN-US">Alternatively you can generate local metrics</span><o:p></o:p></p>
<p><o:p> </o:p></p>
<p><span lang="EN-US">1. Checkout a recent commit to use as the baseline (make sure the working tree is clean)</span><o:p></o:p></p>
<p><span lang="EN-US">2. Run the relevant perf tests</span><o:p></o:p></p>
<p><span lang="EN-US">3. Checkout your branches HEAD commit again</span><o:p></o:p></p>
<p><span lang="EN-US">4. Run the relevant tests again.</span><o:p></o:p></p>
<p><o:p> </o:p></p>
<p><span lang="EN-US">If that doesn't do it, I can have a closer look.</span><o:p></o:p></p>
<p><o:p> </o:p></p>
<p><span lang="EN-US">David E</span><o:p></o:p></p>
<p><o:p> </o:p></p>
<div>
<p class="MsoNormal">On 6/26/20 11:29 AM, Simon Peyton Jones via ghc-devs wrote:<o:p></o:p></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal"><span lang="EN-US">Despite a recent </span><o:p></o:p></p>
<p class="Code"><span lang="EN-US">git fetch <a href="https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.haskell.org%2Fghc%2Fghc-performance-notes.git&data=02%7C01%7Csimonpj%40microsoft.com%7Cc46e95a2b82c42ce4c1608d819c10ecd%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637287664255740875&sdata=aY3P2tJBpL3Ut8RdubHZka0hFtlnr2yhEuNSYA5OfLI%3D&reserved=0">
https://gitlab.haskell.org/ghc/ghc-performance-notes.git</a></span><o:p></o:p></p>
<p class="Code" style="text-indent:13.3pt"><span lang="EN-US">refs/notes/perf:refs/notes/ci/perf</span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US">I’m getting lots of perf regressions in HEAD. For example</span><o:p></o:p></p>
<p class="Code"><span lang="EN-US">=====> T9203(normal) 1 of 1 [0, 0, 0]</span><o:p></o:p></p>
<p class="Code"><span lang="EN-US">]0;T9203(normal) 1 of 1 [0, 0, 0]cd "T9203.run" && "/home/simonpj/code/HEAD-3/inplace/bin/ghc-stage2" -o T9203 T9203.hs -dcore-lint -dstg-lint -dcmm-lint -no-user-package-db -rtsopts -fno-warn-missed-specialisations -fshow-warning-groups
-fdiagnostics-color=never -fno-diagnostics-show-caret -Werror=compat -dno-debug-output -O2<
</span><o:p></o:p></p>
<p class="Code"><span lang="EN-US">cd "T9203.run" && ./T9203 +RTS -V0 -tT9203.stats --machine-readable -RTS <
</span><o:p></o:p></p>
<p class="Code"><span lang="EN-US">runtime/bytes allocated increased from x86_64-linux-deb9 baseline @ HEAD~28:</span><o:p></o:p></p>
<p class="Code"><span lang="EN-US"> Expected T9203 (normal) runtime/bytes allocated: 56046952.0 +/-5%</span><o:p></o:p></p>
<p class="Code"><span lang="EN-US"> Lower bound T9203 (normal) runtime/bytes allocated: 53244604
</span><o:p></o:p></p>
<p class="Code"><span lang="EN-US"> Upper bound T9203 (normal) runtime/bytes allocated: 58849300
</span><o:p></o:p></p>
<p class="Code"><span lang="EN-US"> Actual T9203 (normal) runtime/bytes allocated: 108464536
</span><o:p></o:p></p>
<p class="Code"><span lang="EN-US"> Deviation T9203 (normal) runtime/bytes allocated: 93.5 %</span><o:p></o:p></p>
<p class="Code"><span lang="EN-US">*** unexpected stat test failure for T9203(normal)</span><o:p></o:p></p>
<p class="Code"><span lang="EN-US"> </span><o:p></o:p></p>
<p class="Code"><span lang="EN-US">Performance Metrics (test environment: local):</span><o:p></o:p></p>
<p class="Code"><span lang="EN-US"> </span><o:p></o:p></p>
<p class="Code"><span lang="EN-US">T9203(normal) runtime/bytes allocated 108464536.000</span><o:p></o:p></p>
<p class="Code"><span lang="EN-US"> (baseline @ HEAD~28) 56046952.000 [increased, 93.5%]</span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US">What am I doing wrong?</span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US">Simon</span><o:p></o:p></p>
<p class="MsoNormal" style="margin:0cm;margin-bottom:.0001pt"><span style="mso-fareast-language:EN-GB"><br>
<br>
<o:p></o:p></span></p>
<pre>_______________________________________________<o:p></o:p></pre>
<pre>ghc-devs mailing list<o:p></o:p></pre>
<pre><a href="mailto:ghc-devs@haskell.org">ghc-devs@haskell.org</a><o:p></o:p></pre>
<pre><a href="https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.haskell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-devs&data=02%7C01%7Csimonpj%40microsoft.com%7Cc46e95a2b82c42ce4c1608d819c10ecd%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637287664255740875&sdata=7qu5Ji4KN61LG8RP1X7%2B3GG2kq%2FfSUbnthjV33sYubs%3D&reserved=0">http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs</a><o:p></o:p></pre>
</blockquote>
<pre>-- <o:p></o:p></pre>
<pre>David Eichmann, Haskell Consultant<o:p></o:p></pre>
<pre>Well-Typed LLP, <a href="https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.well-typed.com%2F&data=02%7C01%7Csimonpj%40microsoft.com%7Cc46e95a2b82c42ce4c1608d819c10ecd%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637287664255750861&sdata=VL3fotTvZCGb0wVbrlzmIGHkGEYyqPAlN%2BWd5RRXEN0%3D&reserved=0">http://www.well-typed.com</a><o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>Registered in England & Wales, OC335890<o:p></o:p></pre>
<pre>118 Wymering Mansions, Wymering Road, London W9 2NF, England <o:p></o:p></pre>
</div>
</div>
</body>
</html>