nofib comparisons between 7.0.4, 7.4.2, 7.6.1, and 7.6.2

[Johan Tibell]

Hi all,

I haven't had much time to do performance tzar work yet, but I did run
nofib on the last few GHC releases to see the current trend. The benchmarks
where run on my 64-bit Core i7-3770 @ 3.40GHz Linux machine. Here are the
results:

7.0.4 to 7.4.2:

------------------------------------------------------------
--------------------
        Program           Size    Allocs   Runtime   Elapsed  TotalMem
--------------------------------------------------------------------------------
            Min          -1.6%    -57.3%    -39.1%    -36.4%    -25.0%
            Max         +21.5%   +121.5%    +24.5%    +25.4%   +300.0%
 Geometric Mean          +8.5%     -0.7%     -7.1%     -5.2%     +2.0%

The big loser here in terms of runtime is "kahan", which I added to test
tight loops involving unboxed arrays and floating point arithmetic. I
believe there was a regression in fromIntegral RULES during this release,
which meant that some conversions between fixed-width types went via
Integer, causing unnecessary allocation.

7.4.2 to 7.6.1:

--------------------------------------------------------------------------------
        Program           Size    Allocs   Runtime   Elapsed  TotalMem
--------------------------------------------------------------------------------
            Min          -5.1%    -23.8%    -11.8%    -12.9%    -50.0%
            Max          +5.3%   +225.5%     +7.2%     +8.8%   +200.0%
 Geometric Mean          -0.4%     +2.1%     +0.3%     +0.2%     +0.7%

The biggest loser here in terms of runtime is "integrate". I haven't looked
into why yet.

7.6.1 to 7.6.2:

--------------------------------------------------------------------------------
        Program           Size    Allocs   Runtime   Elapsed  TotalMem
--------------------------------------------------------------------------------
            Min          -2.9%     +0.0%     -4.8%     -4.4%     -1.9%
            Max          +0.0%     +1.0%     +4.5%     +6.4%    +20.8%
 Geometric Mean          -1.7%     +0.0%     +0.1%     +0.3%     +0.2%

I have two takeaways:

 * It's worthwhile running nofib before releases as it does find some
programs that regressed.
 * There are some other regressions out there (i.e. in code on Hackage)
that aren't reflected here, suggesting that we need to add more programs to
nofib.

Cheers,
Johan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/ghc-devs/attachments/20130204/479c17b0/attachment-0001.htm>