[Haskell-cafe] Re: Shootout favouring C

[Brent Fulgham]

--- Sebastian Sylvan <sebastian.sylvan at gmail.com>
wrote:
> I don't think his reaction is reasonable. Especially
> not if he indeed is claiming that the cases where C
is
> favoured are unintentional. The fact that he seems
to take 
> suggestions for improvements as a personal insult
hardly
> helps his case (that the benchmarks aren't
intentionally
> geared to favour C).

Let's not descend into needless ad-hominem attacks.  I
certainly cannot claim that the shootout is perfect,
or that
we do everything right.  But Isaac has been a
long-time
champion of correcting many of the problems in the
shootout
that are largely the result of my ineptitude, and I
think
comments that imply a lack of rigor or desire for
accuracy
are more rightly pointed at me.
 
> Daniel's benchmarks are interesting, and instead of
> a paranoid and rude response from the people
responsible,
> I would rather have liked to see if the results are
the
> same on the systems used in the shootout.

I think I answered this in another e-mail.  I do see
similar
results on the shootout machine.  Out of a desire for
inclusiveness, we arbitrarily kept lower values of N
so that
certain scripting languages (mainly Ruby and Python)
would
not self-destruct when attempting to process larger
values
of N.

Furthermore, I had arbitrarily restricted the timeout
to 600 seconds to avoid running the shootout for weeks
due
to poor performance in some implementations.  However,
now 
that the main shootout tests are stabilized, it's not
such
a big deal to extend the timeout (as we have done for
spectralnorm and others), and I think it would be good
to
do so for the Ackermann test.

Thanks,

-Brent