[Haskell-cafe] Re: [Haskell] [Fwd: Re: Computer Language Shootout]

[Neil Mitchell]

Hi

> I have to disagree with this. That is, I don't object to Don's
> explanation of why the shootout entries degraded in this particular
> case, but I do think that Andrzej was right to point this out:
> "Perhaps making a collective effort towards benchmarking Haskell programs and
> analyzing the results in some methodic way could prove helpful?"
> and I think that he *is* pointing out a fundamental issue here.

We have the nofib suite, it could do with some attention, but it is
(or was) a pretty good performance tester.

> Maybe it's just me, but as someone who has some amount of experience
> with implementing optimizations for Haskell, I find it nearly
> impossible to precisely understand and measure exactly how those
> optimizations improve (or degrade) program performance.

The problem is that something like GHC is very complex, with lots of
transformations. When transformations are firing other
transformations, which in turn fire other transformations, it doesn't
take a great deal to disrupt this flow of optimisation and end up with
a result which doesn't accurately reflect the particular change you
made. Better knowledge of the end effects on a program isn't
necessarily going to translate to better knowledge of the
optimisations effect.

Maybe if we had a greater number and variety of optimising compilers
we'd be able to more freely experiment with optimisation techniques in
different settings. At the moment GHC is all there is (with Jhc not
ready for mainstream use yet)

Thanks

Neil