[Haskell-cafe] proposal: HaBench, a Haskell Benchmark Suite

[David Roundy]

On Sun, Jan 28, 2007 at 10:36:50PM +0100, Andy Georges wrote:
> On 28 Jan 2007, at 12:57, Joel Reymont wrote:
> >On Jan 28, 2007, at 8:51 AM, Andy Georges wrote:
> >
> >>it is nice to know that e.g., Data.ByteString performs as good as  
> >>C, but is would be even nicer to see that large, real-life apps  
> >>can reach that same performance.
> >
> >What about using darcs as a benchmark? I heard people say it's  
> >slow. The undercurrent is that it's slow because it's written in  
> >Haskell.
> 
> I have pondered about that. What would the input set be? And how to  
> repeatedly run the benchmark? Should we just have a recording phase?  
> Or a diff phase? It seems difficult to have a VC system as a benchmark.

We darcs folk would love to have a darcs benchmark, and Jason has even put
some work into a simulator driver (which would call various darcs commands
to build up a repository).  But I don't think this would be useful as a
Haskell benchmark.  But we'd love to have automatic performance regression
testing!

Better (for Haskell benchmarking) would be to take a small (but relevant)
part of darcs, and benchmark that.  Which is effectively what a
Data.ByteString benchmark could do.  Darcs spends vast amounts of time
breaking huge files into lines (or it's been doing that for me recently)
and running its LCS algorithm on those lists of lines.  This wouldn't be a
bad benchmark: just a huge LCS job.  You could algorithmically generate two
huge strings and then compute their LCS.  Actually, darcs no longer uses a
true LCS (neither does diff), so you might try our LCS substitute.
-- 
David Roundy
http://www.darcs.net