performance monitoring idea

[Richard Eisenberg]

Hi all,

I'm deep into working on #9233 right now, which may stem from various inefficiencies due to roles. I'm making good progress, and I feel badly about introducing these problems. When I was coding this originally, I was thinking "Premature optimization is the root of all evil" and didn't pay too much attention to performance, thinking that a perf test would show me up if I erred. I was wrong.

This all got me thinking: could we track some cumulative timing numbers for running the whole testsuite? As a first draft, it could just be the sum of the MUT statistic from +RTS -t. As part of our growing CI support (thanks, Joachim and others!!), we could track this number over time. When a commit slows GHC down, we would hopefully see it reflected here and then try to catch these bugs earlier, in an effort to defy Lennart's expectations of GHC slowing down every release. (Sorry, can't find reference to quote at the moment.)

This idea seems easy enough to implement. It may not be terribly reliable, but it might just be reliable enough to help catch these problems before releases and to help to discover where they came from.

What do we think?

Richard