[Haskell-cafe] proposal: HaBench, a Haskell Benchmark Suite

[Kenneth Hoste]

Hello,

Following up and the threads on haskell and haskell-cafe, I'd like to  
gather ideas, comments and suggestions for a standarized Haskell  
Benchmark Suite.

The idea is to gather a bunch of programs written in Haskell, and  
which are representative for the Haskell community (i.e. apps,  
libraries, ...). Following the example of SPEC (besides the fact that  
the SPEC benchmarks aren't available for free), we would like to  
build a database containing performance measurements for the various  
benchmarks in the suite. Users should be able to submit their  
results. This will hopefully stimulate people to take performance  
into account when writing a Haskell program/library, and will also  
serve as a valuable tool for further optimizing both applications  
written in Haskell and the various Haskell compilers out there (GHC,  
jhc, nhc, ...).

This thread is meant to gather peoples thought on this subject.
Which programs should we consider for the first version of the  
Haskell benchmark suite?
How should we standarize them, and make them produce reliable  
performance measurement?
Should we only use hardware performance counters, or also do more  
thorough analysis such as data locality studies, ...
Are there any papers available on this subject (I know about the  
paper which is being written as we speak ICFP, which uses PAPI as a  
tool).

I have created a HaskellWiki page (http://www.haskell.org/haskellwiki/ 
HaBench) in order to centralize ideas and suggestions. Feel free to  
add anything, and if you're willing to contribute to this project (in  
any way), add your name to the wiki for future reference.

greetings,

Kenneth (a.k.a. boegel)

-- 

Statistics are like a bikini. What they reveal is suggestive, but  
what they conceal is vital (Aaron Levenstein)

Kenneth Hoste
ELIS - Ghent University
kenneth.hoste at elis.ugent.be
http://www.elis.ugent.be/~kehoste