[Haskell-cafe] Haskell performance

[Simon Peyton-Jones]

Don, and others,

This thread triggered something I've had at the back of my mind for some time.

The traffic on Haskell Cafe suggests that there is a lot of interest in the performance of Haskell programs.  However, at the moment we don't have any good *performance* regression tests for GHC. We have zillions of behavioural regression tests (this program should compile, this one should fail), but nothing much on performance. We have the nofib suite, but it's pretty static these days.  Peter's set of benchmarks are great (if very specific to strings etc, but that's fine), and it'd be a pity of they now sink beneath the waves.

What would be v helpful would be a regression suite aimed at performance, that benchmarked GHC (and perhaps other Haskell compilers) against a set of programs, regularly, and published the results on a web page, highlighting regressions.  Kind of like the Shootout, only just for Haskell, and with many more programs.

Like Hackage, it should be easy to add a new program.  It'd be good to measure run-time, but allocation count, peak memory use, code size, compilation time are also good (and rather more stable) numbers to capture.

Does anyone feel like doing this?  It'd be a great service.  No need to know anything much about GHC.

Simon

| -----Original Message-----
| From: haskell-cafe-bounces at haskell.org [mailto:haskell-cafe-bounces at haskell.org] On Behalf Of Don Stewart
| Sent: 16 December 2007 23:22
| To: Peter Lund
| Cc: haskell-cafe
| Subject: Re: [Haskell-cafe] [RFC] benchmarks of bytestrings, teaser
|
| I've had a look at how some of the code was being compiled for
| strict and lazy bytestrings, and also which rules weren't firing.
| With some small tweaks the code seems back in good shape.