[Haskell-cafe] Re: Haskell performance

[Simon Marlow]

Malcolm Wallace wrote:
> Simon Peyton-Jones <simonpj at microsoft.com> wrote:
> 
>> What would be v helpful would be a regression suite aimed at
>> performance, that benchmarked GHC (and perhaps other Haskell
>> compilers) against a set of programs, regularly, and published the
>> results on a web page, highlighting regressions.
> 
> Something along these lines already exists - the nobench suite.
>     darcs get http://www.cse.unsw.edu.au/~dons/code/nobench
> It originally compared ghc, ghci, hugs, nhc98, hbc, and jhc.
> (Currently the results at
>     http://www.cse.unsw.edu.au/~dons/nobench.html
> compare only variations of ghc fusion rules.)
> 
> I have just been setting up my own local copy - initial results at
>     http://www.cs.york.ac.uk/fp/nobench/powerpc/results.html
> where I intend to compare ghc from each of the 6.4, 6.6 and 6.8
> branches, against nhc98 and any other compilers I can get working.
> I have powerpc, intel, and possibly sparc machines available.

That's great.  BTW, GHC has a performance bug affecting calendar at the moment:

   http://hackage.haskell.org/trac/ghc/ticket/1168

The best GHC options for this program might therefore be -O2 
-fno-state-hack.  Or perhaps just -O0.

>> Like Hackage, it should be easy to add a new program.
> 
> Is submitting a patch against the darcs repo sufficiently easy?
> Should we move the master darcs repo to somewhere more accessible, like
> code.haskell.org?

Yes, please do.  When I have a chance I'd like to help out.

>> It'd be good to measure run-time,
> 
> Done...
> 
>> but allocation count, peak memory use, code size,
>> compilation time are also good (and rather more stable) numbers to
>> capture.
> 
> Nobench does already collect code size, but does not yet display it in
> the results table.  I specifically want to collect compile time as well.
> Not sure what the best way to measure allocation and peak memory use
> are?

With GHC you need to use "+RTS -s" and then slurp in the <prog>.stat file. 
  You can also get allocations, peak memory use, and separate mutator/GC 
times this way.

Cheers,
	Simon