[Haskell-cafe] Moving nobench towards HaBench

[Andy Georges]

Hello,


A while back, we had a discussion on #haskell about assembling a  
Haskell benchmark suite, that is suitable for doing performance tests.  
A preliminary page was erected athttp://www.haskell.org/haskellwiki/HaBench 
. In the meantime, Donald Steward extended the original nofib suite  
with some shootout benchmarks afaik, resulting in nobench. The code  
base for the latter currently resides at http://code.haskell.org/nobench/ 
.

I have been trying to get it running on GHC 6.10.1. For now, I added a  
number of type definitions to the code, causing the build/runtime  
system to compile. The same probably ought to be done for the  
benchmarks themselves, unless there is a cheat around this using some  
language extension. Anyhow, I'll post a patch against the current  
repository as soon as I have a number of benchmarks running.

The main issue that still remains is the availability of real life  
benchmarks. I agree with the fact that micro-benchmarks can be useful  
for testing purposes or measures the efficacy and effectiveness of  
certain optimisations, yet I firmly believe any community has need of  
a set of benchmarks that actually reflects the real life usage of the  
language. I am think along the lines of something alike to the DaCapo  
projects, which assembled a number of very good benchmarks for the  
Java language and its VM. So the question basically boils down to  
this. Is there anybody interested in making the move toward HaBench,  
and if so, do you know of real life benchmarks that can serve for this  
exact purpose?

The benchmarks should preferably execute for > 10s on modern machines,  
using a decent amount of RAM (say somewhere between 50 and 500MB),  
thus exercising all parts of a modern computing system. The code  
should not be trivial and the set of benchmarks should eventually  
cover the most uses of Haskell in the industry. Of course, the  
benchmarks themselves should be open source. If possible, they should  
come with multiple inputs, allowing a short (test) run as well as  
longer measurement runs.


If you are able and willing to help out, drop by at the HaBench page  
and drop a line,


-- Andy