Haskell performance

[Josef Svenningsson]

On Thu, 18 Mar 2004, Carsten Schultz wrote:

> Hi Sébastien!
>
> On Thu, Mar 18, 2004 at 11:30:26AM +0100, Sébastien Pierre wrote:
> > In fact, I would like to know how Haskell compares in performance to
> > other languages because if I refer to the page I mentioned
> > (http://www.bagley.org/~doug/shootout/craps.shtml) it does not even
> > compete with Python (which is rather... slow).
>
> You should look at the individual examples and see how relevant their
> results are for you.  And keep in mind that strings tend to be slow in
> Haskell (being lazy lists of characters), a fact that may influence
> some of the tests.
>
I agree completely. Here's a little more detailed analysis to make the
comparison fair.

The language shootout has 25 benchmarks to compare languages with. One
thing which is not Haskell's in favour is the fact that Haskell isn't
represented in all of these benchmarks. So to get a better comparison we
should set the score to zero on those benchmarks where we don't have a
Haskell program. I zeroed out the following benchmarks:
List Processing, Method Calls, Object Instantiation, String Concatenation,
Hashes Part II, and Regular Expression Matching.

Suddenly Haskell climbs from 19'th place to 14'th place. Not dramatic but
at least the comparison is more fair now. It is also interesting to see
what happens if you set the lines of code multiplier to one instead of
zero. Haskell now gets 7'th place. Just before Python as it happens....

Well, I think this shows that one should be very careful when reading
these kinds of benchmarks. It is very easy to jump to conclusions. But I
do believe there are some conclusions to be drawn if we look close enough.

The main problem for Haskell on these benchmarks is I/O. In all benchmarks
with I/O in them Haskell (or I should really say ghc) gets a really bad
score. This is, however, a known problem. Simon PJ mentioned it in a
previous mail. All I can say is that for my purposes it hasn't been a
problem, so I really don't care about these benchmarks.

Another thing with these benchmarks is that some of them are so tightly
specified that the idiomatic Haskell solution didn't fit in. See for
example the String Concatentation benchmark. The programs have to
concatenate the string in such a way that the Haskell program gets a
quadratic behaviour. I submitted a solution using concat which behaved
OK but it was not allowed to compete. You can find it on the page of the
benchmark.

Well, enough about benchmarks. Just make sure you read them carefully.

S'ebastien, I encourage you to choose the Haskell + C solution. It has
worked well for me and many others. Paradox, a model finder for first
order logic is written in Haskell plus some C for the performance
sensitive parts. It won the SAT/Models class of last years CASC
competition. There only speed counts (after correctness ofcourse). See:
http://www.cs.chalmers.se/~koen/paradox/


All the best,

	/Josef