[Haskell-cafe] Re: proposal: HaBench, a Haskell Benchmark Suite

[Chris Kuklewicz]

Al Falloon wrote:
> David Roundy wrote:
>> On Fri, Jan 26, 2007 at 10:17:28AM -0500, Al Falloon wrote:
>>> Kenneth Hoste wrote:
>>>> The idea is to gather a bunch of programs written in Haskell, and
>>>> which are representative for the Haskell community (i.e. apps,
>>>> libraries, ...).
>>> A While ago I tried to write a Haskell version of John Harrops
>>> ray-tracer benchmark
>>> (http://www.ffconsultancy.com/free/ray_tracer/languages.html) but the
>>> performance was not very good (the OCaml version I based it on was at
>>> least 10x faster).
>>>
>>> I would be happy to contribute my code to the benchmark suite if you
>>> are interested. Perhaps someone can point out obvious speed-ups that
>>> I missed while trying to improve the performance.
>>
>> I would think that what we'd want to benchmark would be clean, optimized
>> actually-used code.  I.e. things like Data.Bytestring, so that we
>> could see
>> how compilers differed on important code, or how the code generated on
>> different architectures differed.  e.g. if jhc beats ghc on amd64, the
>> ghc
>> developers would probably be very curious as to why, and how to fix itere 
>>
>> Code that's not been properly optimized with respect to strictness, etc,
>> would fail to focus the tests on important optimizations of the compiler.
>> But of course, the benchmark code should also be clean, since we want to
>> ensure that our compilers are good enough that we can write useful
>> beautiful code that is also fast.
> 
> I tried to optimize it, but I couldn't approach the speed of the OCaml
> version. I followed the performance tuning advice from the Wiki, and had
> even resorted to writing the inner loop calculations using all unboxed
> doubles, but without significant improvements. This is exactly the kind
> of code that I write most often, and I would love to see improvements in
> the optimizations for this kind of numerically intensive code
> (especially without having to resort to compiler-specific unboxed
> representations).
> 
> I agree that common libraries like ByteString need to be well
> represented, but the original request additionally included programs
> that are representative of applications. A ray-tracer (even with a fixed
> scene and only one type of scene primitive) is a fairly nice
> approximation of a real numerical batch-oriented application while still
> being small enough to understand and modify. I expect thats why Jo
> chose it as his benchmark application in the first place.

Writing numeric code that processes Doubles is hard to optimize.  See the
shootout example for the n-body problem:

http://haskell.org/haskellwiki/Shootout/Nbody
http://shootout.alioth.debian.org/debian/benchmark.php?test=nbody&lang=all

Some of the best work on making Haskell perform wonderfully at numerics is
summarized at http://haskell.org/haskellwiki/GHC/Data_Parallel_Haskell
and their status report http://www.cse.unsw.edu.au/~chak/papers/CLPKM06.html