GHC perf

[Simon Peyton Jones]

We store the metrics in git notes *per-commit*. All metrics for commit XX are stored on the git note for commit XX. You can even view the raw data with this command (where XX is the commit hash):
OK.   But the master repo *already* has perf notes for that commit (I assume).  Do mine somehow overwrite the master copy?

So suppose, on my local machine, I do

$ git checkout a12b34c56 && git submodule update --init

$ ./hadrian/build.sh test --only-perf
Now you say that I'm going to create git notes for a12b34c56.  But those are purely for my local machine!  Maybe my compiler is build with -DDEBUG.  I don't want them to accidentally land in the main repo as the canonical perf figures for a12b34c56.

How do I avoid accidentally pushing them?

I should stress one caveat: we do not save metrics if you have uncommitted changes.
Oh wow.  Put that in MASSIVE BOLD CAPITALS.   You mean that the entire exercise will (silently) be bogus if I have any uncommitted changes?   That's a bit of a pain if I make a change, run some perf tests, make another change, run again.  But I can live with it if I know.

Simon


From: David Eichmann <davide at well-typed.com>
Sent: 23 January 2020 14:48
To: Simon Peyton Jones <simonpj at microsoft.com>
Subject: Re: GHC perf

Which wiki page?
https://gitlab.haskell.org/ghc/ghc/wikis/building/running-tests/performance-tests<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.haskell.org%2Fghc%2Fghc%2Fwikis%2Fbuilding%2Frunning-tests%2Fperformance-tests&data=02%7C01%7Csimonpj%40microsoft.com%7C289a849201d9409910ee08d7a0133e0e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637153876826944492&sdata=rSV1sowxBS6%2FZhEQu6KLvM5Hsm%2F3xNT49MsL2YWsBRE%3D&reserved=0>
Ah.  Now I'm lost.  Somehow the second and fourth line must be recording info, locally in my tree, but two distinct batches of information.   Perhaps kept distinct by the current commit?  Where is the info actually stored?

All metric results are stored in git notes. This is a feature of git that lets you attach arbitrary text to a commit (without affecting the commit's hash). It's mentioned here<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.haskell.org%2Fghc%2Fghc%2Fwikis%2Fbuilding%2Frunning-tests%2Fperformance-tests%23performance-metrics-are-logged&data=02%7C01%7Csimonpj%40microsoft.com%7C289a849201d9409910ee08d7a0133e0e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637153876826944492&sdata=%2FfkL0GwLNr3Eicyi%2FmI%2Fl%2F747BsAr8mHrbxzfRNkSSU%3D&reserved=0>. Whenever you run a performance test, the raw metrics will be appended to the git note for the current commit in a simple tab separated value (tsv) format.
OK, suppose I start from commit XX, and make some local changes.   Then I do the -only-perf thing.  presumably that'll be recorded tagged with XX.  That's fine; just want it to be clear.  Worth adding this info to the wiki page, so we have a clear mental model.

We store the metrics in git notes *per-commit*. All metrics for commit XX are stored on the git note for commit XX. You can even view the raw data with this command (where XX is the commit hash):

$ git notes --ref perf show XX

NOTE `--only-perf` is optional. It limits the test runner to only run performance tests but the performance metrics will be stored regardless of this option. So, if you've ever run performance test locally, chances are the metrics will have be record without you even knowing.

I should stress one caveat: we do not save metrics if you have uncommitted changes.

--

David Eichmann, Haskell Consultant

Well-Typed LLP, http://www.well-typed.com<https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.well-typed.com%2F&data=02%7C01%7Csimonpj%40microsoft.com%7C289a849201d9409910ee08d7a0133e0e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637153876826954448&sdata=3QRZZZro5aN4kYVEsOb1r7VDtdy7IxPbToMA9wPiQrc%3D&reserved=0>



Registered in England & Wales, OC335890

118 Wymering Mansions, Wymering Road, London W9 2NF, England
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20200123/906eb6e5/attachment-0001.html>