<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
pre
{mso-style-priority:99;
mso-style-link:"HTML Preformatted Char";
margin:0cm;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New",serif;}
span.HTMLPreformattedChar
{mso-style-name:"HTML Preformatted Char";
mso-style-priority:99;
mso-style-link:"HTML Preformatted";
font-family:Consolas;}
span.EmailStyle22
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-GB" link="blue" vlink="purple">
<div class="WordSection1">
<p style="margin-left:36.0pt">We store the metrics in git notes *per-commit*. All metrics for commit XX are stored on the git note for commit XX. You can even view the raw data with this command (where XX is the commit hash):<o:p></o:p></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US">OK. But the master repo *<b>already</b>* has perf notes for that commit (I assume). Do mine somehow overwrite the master copy?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US">So suppose, on my local machine, I do<o:p></o:p></span></p>
<pre style="margin-left:36.0pt">$ git checkout <b>a12b34c56</b> && git submodule update --init<o:p></o:p></pre>
<pre style="margin-left:36.0pt">$ ./hadrian/build.sh test --only-perf<o:p></o:p></pre>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US">Now you say that I’m going to create git notes for
</span>a12b34c56. But those are purely for my local machine! Maybe my compiler is build with -DDEBUG. I don’t want them to accidentally land in the main repo as the canonical perf figures for
<b>a12b34c56</b>.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">How do I avoid accidentally pushing them?<o:p></o:p></p>
<p style="margin-left:36.0pt">I should stress one caveat: we do not save metrics if you have uncommitted changes.<o:p></o:p></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US">Oh wow. Put that in MASSIVE BOLD CAPITALS. You mean that the entire exercise will (silently) be bogus if I have any uncommitted changes? That’s a bit of a pain if I make a change, run some perf
tests, make another change, run again. But I can live with it if I know.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US">Simon<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0cm 0cm 0cm 4.0pt">
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span lang="EN-US">From:</span></b><span lang="EN-US"> David Eichmann <davide@well-typed.com>
<br>
<b>Sent:</b> 23 January 2020 14:48<br>
<b>To:</b> Simon Peyton Jones <simonpj@microsoft.com><br>
<b>Subject:</b> Re: GHC perf<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal"><i>Which</i> wiki page?<o:p></o:p></p>
</blockquote>
<p class="MsoNormal"><a href="https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.haskell.org%2Fghc%2Fghc%2Fwikis%2Fbuilding%2Frunning-tests%2Fperformance-tests&data=02%7C01%7Csimonpj%40microsoft.com%7C289a849201d9409910ee08d7a0133e0e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637153876826944492&sdata=rSV1sowxBS6%2FZhEQu6KLvM5Hsm%2F3xNT49MsL2YWsBRE%3D&reserved=0">https://gitlab.haskell.org/ghc/ghc/wikis/building/running-tests/performance-tests</a>
<o:p></o:p></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Ah. Now I’m lost. Somehow the second and fourth line must be recording info, locally in my tree, but two distinct batches of information. Perhaps kept distinct by the current
commit? Where is the info actually stored?<o:p></o:p></p>
</blockquote>
<p>All metric results are stored in git notes. This is a feature of git that lets you attach arbitrary text to a commit (without affecting the commit's hash). It's mentioned
<a href="https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.haskell.org%2Fghc%2Fghc%2Fwikis%2Fbuilding%2Frunning-tests%2Fperformance-tests%23performance-metrics-are-logged&data=02%7C01%7Csimonpj%40microsoft.com%7C289a849201d9409910ee08d7a0133e0e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637153876826944492&sdata=%2FfkL0GwLNr3Eicyi%2FmI%2Fl%2F747BsAr8mHrbxzfRNkSSU%3D&reserved=0">
here</a>. Whenever you run a performance test, the raw metrics will be appended to the git note for the current commit in a simple tab separated value (tsv) format.<o:p></o:p></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">OK, suppose I start from commit XX, and make some local changes. Then I do the –only-perf thing. presumably that’ll be recorded tagged with XX. That’s fine; just want it to
be clear. Worth adding this info to the wiki page, so we have a clear mental model.<o:p></o:p></p>
</blockquote>
<p>We store the metrics in git notes *per-commit*. All metrics for commit XX are stored on the git note for commit XX. You can even view the raw data with this command (where XX is the commit hash):<o:p></o:p></p>
<p>$ git notes --ref perf show XX<o:p></o:p></p>
<p>NOTE `--only-perf` is optional. It limits the test runner to only run performance tests but the performance metrics will be stored regardless of this option. So, if you've ever run performance test locally, chances are the metrics will have be record without
you even knowing.<o:p></o:p></p>
<p>I should stress one caveat: we do not save metrics if you have uncommitted changes.<o:p></o:p></p>
<pre>-- <o:p></o:p></pre>
<pre>David Eichmann, Haskell Consultant<o:p></o:p></pre>
<pre>Well-Typed LLP, <a href="https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.well-typed.com%2F&data=02%7C01%7Csimonpj%40microsoft.com%7C289a849201d9409910ee08d7a0133e0e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637153876826954448&sdata=3QRZZZro5aN4kYVEsOb1r7VDtdy7IxPbToMA9wPiQrc%3D&reserved=0">http://www.well-typed.com</a><o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>Registered in England & Wales, OC335890<o:p></o:p></pre>
<pre>118 Wymering Mansions, Wymering Road, London W9 2NF, England <o:p></o:p></pre>
</div>
</div>
</body>
</html>