Perf test viewer

[Ben Gamari]

Sylvain Henry <sylvain at haskus.fr> writes:

> Hello everyone,
>
> Since testsuite performance results are stored into Git notes, they are 
> more difficult to visualize. At least with values in .T files we could 
> see the changes over time, but now increase/decrease are only indicated 
> into commit messages (but not necessarily by how much). The only tool we 
> have afaik is the perf_notes.py script [1] but it's not very interactive.
>
> So, long story short, I've started another one which is more interactive 
> (in the browser). An instance is running on my server: http://hsyl20.fr:4222
>
Hi Sylvain,

This looks great!

I would also note that all of the performance metrics produced by the
testsuite, nofib, and (soon) head.hackage are preserved in a PostgreSQL
database on gitlab.haskell.org which can be accessed by way of Postgrest
[1]. The schema can be found here [2]. You will also find the import
tools in the same repository (but they are quite rough around the edges;
you have been warned).

The schema is probably most conveniently used via the `results_view`
view. For instance, one can request all metrics for commits in the last
month with:

    $ curl https://perf.ghc.haskell.org/db/results_view?commit_date=gt.2020-09-01

Another useful view is `deltas`, which provides compares metrics for
pairs of commits. This I have found quite useful in spotting
regressions.

Apart from curl, there are a few other options for consuming this
information:

 * I have an Python notebook which has some convenient helpers for
   quickly plotting things in `ghc-utils` [3]. I find this option to
   have the greatest power/weight ratio for most use-cases.

 * There is an incredibly hacky attempt at a web interface here [4]
   (source here [5]) although it's probably best to bury this sad effort
  
 * Sometimes I have needed to fall back to SQL for some queries; if
   someone would find this useful we can arrange read-only SQL access on
   an individual basis.

Regardless, this project has been gradually evolving as the need arises
for the last few years. It's been quite useful but I think it's
potential is quite under-realized.

Cheers,

- Ben


[1] https://postgrest.org/
[2] https://github.com/bgamari/ghc-perf-import/blob/master/import/schema.sql
[3] https://github.com/bgamari/ghc-perf-import/blob/master/plot.ipynb
[4] http://home.smart-cactus.org/ghc-perf/
[5] https://github.com/bgamari/ghc-perf-import/tree/master/web
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20201015/d54bd7c6/attachment.sig>