<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
</head>
<body>
<p><br>
</p>
<div class="moz-forward-container"><br>
<br>
-------- Forwarded Message --------
<table class="moz-email-headers-table" cellspacing="0"
cellpadding="0" border="0">
<tbody>
<tr>
<th valign="BASELINE" nowrap="nowrap" align="RIGHT">Subject:
</th>
<td>Re: performance testing</td>
</tr>
<tr>
<th valign="BASELINE" nowrap="nowrap" align="RIGHT">Date: </th>
<td>Wed, 11 Mar 2020 12:08:52 +0000</td>
</tr>
<tr>
<th valign="BASELINE" nowrap="nowrap" align="RIGHT">From: </th>
<td>David Eichmann <a class="moz-txt-link-rfc2396E" href="mailto:davide@well-typed.com"><davide@well-typed.com></a></td>
</tr>
<tr>
<th valign="BASELINE" nowrap="nowrap" align="RIGHT">To: </th>
<td>Richard Eisenberg <a class="moz-txt-link-rfc2396E" href="mailto:rae@richarde.dev"><rae@richarde.dev></a></td>
</tr>
</tbody>
</table>
<br>
<br>
<br>
<blockquote type="cite">Ah yes -- very helpful. So the baseline is
always (implicitly) the parent commit, if that commit has
performance info in the set of notes. So if I have this
situation (time flows down)<br>
<br>
origin/master: blah<br>
wip/xyz: big cool change (that slows things down)<br>
attempt1: comment out some of big cool change<br>
attempt2: comment out more of big cool change<br>
</blockquote>
With you so far.<br>
<blockquote type="cite">and I do a perf test on attempt1, it's
quite likely that the perf test will*pass*, because it's
comparing to the previous commit.<br>
</blockquote>
Yes, assuming you actually ran performance tests on the previous
commit with a clean working tree (remember we don't record
performance metrics if the git tree has changes, though you'll see
a warning in the test output in that case). I'd suggest outputting
a graph as described in the wiki if you're wondering which commits
have recorded performance metrics.<br>
<blockquote type="cite"> And then when I do a perf test on
attempt2 and see a metric*decrease*, that might just be because
I've fixed the perf problem... but I haven't actually made an
improvement.<br>
</blockquote>
<br>
Right. The decrease is compared to attempt1, but this may not be a
decrease compared to earlier commits. Again I'd suggest graphing
the data if you want to inspect this yourself.<br>
<br>
<br>
David E<br>
<br>
<br>
<br>
<pre class="moz-signature">--
David Eichmann, Haskell Consultant
Well-Typed LLP, <a class="moz-txt-link-freetext" href="http://www.well-typed.com">http://www.well-typed.com</a>
Registered in England & Wales, OC335890
118 Wymering Mansions, Wymering Road, London W9 2NF, England
</pre>
</div>
</body>
</html>