GHC's performance

[Johan Tibell]

There are a few "projects" in this area I think we should undertake:

## Set up a performance dashboard

Example: http://goperfd.appspot.com/ (try clicking the "Perf" and "Graphs"
buttons.)

I think the way to go here is:

 * Figure out how to build GHC and run the nofib suite in a repeatable
(i.e. scripted) manner.
 * Set up a Jenkins build bot on a "quiet" machine. Have it run the above
script in "exclusive" mode (i.e. no other Jenkins jobs can run in parallel).
 * Write a script that gathers the nofib output and sticks it in a
database. The database should be keyed by the commit hash. Use a mature DB
(e.g. MySQL, sqlite, postgress).
 * Write a little web frontend that graphs the results over time. Don't
reinvent the wheel/shave all the Yaks. You might be able to reuse the Go
frontend and run it on appengine.
 * Bonus points: have the jenkins job send an email if performance
regresses above some threshold.

There might already exist Jenkins plugins that can help with the last 4
steps.

## Improve our benchmarks

Many of the benchmarks in nofib have too short runtime nowadays to be
accurate enough when comparing the performance of two GHC builds. The
shootout benchmarks are good (I've checked), but the remaining ones should
be considered suspicious.

We ought to weed out or improve benchmarks that are no longer accurate.
This might involve having the run on bigger inputs and thus run for longer.

In addition, running the existing benchmark suites of some core libraries
(e.g. from the HP) would also be very useful. The difficulty is to get
Criterion to build with HEAD reliably.

## One-off nofib run against the lastest N major GHC releases

To see if we have already regressed (and see where we have regressed) I
think we should just run the current nofib suite (probably using the "slow"
mode) against 6.12 and up to see where we have already regressed. File a
bug for each regression, optionally with a small analysis of why we
regressed (e.g. look at the Core and the output of +RTS -s).

Cheers,
  Johan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/ghc-devs/attachments/20140410/305ef3f2/attachment.html>