[Haskell-cafe] proposal: HaBench, a Haskell Benchmark Suite

[Simon Peyton-Jones]

| I think that we should have, as David Roundy pointed out, a
| restriction to code that is actually used frequently. However, I
| think we should make a distinction between micro-benchmarks, that
| test some specific item, and real-life benchmarks.

As many of you will know, the nofib benchmark suite made just such a distinction from the beginning.  In fact, it has 3 sub-suites:

* Imaginary: very tiny programs whose merit is that they sometimes hit missing optimisations very hard indeed.  Useful for compiler writers.

* Spectral: these are often called "kernels" and are meant to be typical of the inner loops of some real programs.

* Real: these are unadulterated real applications, of various sizes.

We found these categories to be useful and robust, and I think they'd be useful for the new suite.  In particular, the imaginary suite is useless for (say) choosing a compiler, but fantastic for exposing particular weak spots.   But if the imaginary programs were mixed with the real ones, the whole thing would lose credibility.

You can find more info here:
        http://research.microsoft.com/~simonpj/Papers/nofib.ps

Simon