nofib comparisons between 7.0.4, 7.4.2, 7.6.1, and 7.6.2

[David Terei]

On 5 February 2013 01:24, Nicolas Frisby <nicolas.frisby at gmail.com> wrote:
> Is anyone familiar with the "fibon" directory within the nofib.git
> repository?
>
> http://darcs.haskell.org/nofib/fibon/

Yes. They are from here: https://github.com/dmpots/fibon

Fibon is a newer, alternative benchmarking suite for Haskell done by
David M Peixotto. I've used it at times but sadly haven't had much
luck, it always seems to take many hours to run on my machine.

>
> Johan, this at least seems like an potential home for the additional
> programs you suggested adding. In particular, it has Repa, Dph, Shootout,
> and Hackage subdirectories.
>
> I'm doing a GHC HQ internship at the moment, and one of the
> just-needs-to-happen tasks on my (growing) todo list is to look into fibon.
>
> SPJ recalls that not all of the various building infrastructures were
> getting along. Anyone know the story? Thanks!
>
>
> On Mon, Feb 4, 2013 at 10:33 PM, Johan Tibell <johan.tibell at gmail.com>
> wrote:
>>
>> Hi all,
>>
>> I haven't had much time to do performance tzar work yet, but I did run
>> nofib on the last few GHC releases to see the current trend. The benchmarks
>> where run on my 64-bit Core i7-3770 @ 3.40GHz Linux machine. Here are the
>> results:
>>
>> 7.0.4 to 7.4.2:
>>
>>
>> --------------------------------------------------------------------------------
>>         Program           Size    Allocs   Runtime   Elapsed  TotalMem
>>
>> --------------------------------------------------------------------------------
>>             Min          -1.6%    -57.3%    -39.1%    -36.4%    -25.0%
>>             Max         +21.5%   +121.5%    +24.5%    +25.4%   +300.0%
>>  Geometric Mean          +8.5%     -0.7%     -7.1%     -5.2%     +2.0%
>>
>> The big loser here in terms of runtime is "kahan", which I added to test
>> tight loops involving unboxed arrays and floating point arithmetic. I
>> believe there was a regression in fromIntegral RULES during this release,
>> which meant that some conversions between fixed-width types went via
>> Integer, causing unnecessary allocation.
>>
>> 7.4.2 to 7.6.1:
>>
>>
>> --------------------------------------------------------------------------------
>>         Program           Size    Allocs   Runtime   Elapsed  TotalMem
>>
>> --------------------------------------------------------------------------------
>>             Min          -5.1%    -23.8%    -11.8%    -12.9%    -50.0%
>>             Max          +5.3%   +225.5%     +7.2%     +8.8%   +200.0%
>>  Geometric Mean          -0.4%     +2.1%     +0.3%     +0.2%     +0.7%
>>
>> The biggest loser here in terms of runtime is "integrate". I haven't
>> looked into why yet.
>>
>> 7.6.1 to 7.6.2:
>>
>>
>> --------------------------------------------------------------------------------
>>         Program           Size    Allocs   Runtime   Elapsed  TotalMem
>>
>> --------------------------------------------------------------------------------
>>             Min          -2.9%     +0.0%     -4.8%     -4.4%     -1.9%
>>             Max          +0.0%     +1.0%     +4.5%     +6.4%    +20.8%
>>  Geometric Mean          -1.7%     +0.0%     +0.1%     +0.3%     +0.2%
>>
>> I have two takeaways:
>>
>>  * It's worthwhile running nofib before releases as it does find some
>> programs that regressed.
>>  * There are some other regressions out there (i.e. in code on Hackage)
>> that aren't reflected here, suggesting that we need to add more programs to
>> nofib.
>>
>> Cheers,
>> Johan
>>
>>
>> _______________________________________________
>> ghc-devs mailing list
>> ghc-devs at haskell.org
>> http://www.haskell.org/mailman/listinfo/ghc-devs
>>
>
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://www.haskell.org/mailman/listinfo/ghc-devs
>