nofib comparisons between 7.0.4, 7.4.2, 7.6.1, and 7.6.2

[Simon Marlow]

On 05/02/13 10:13, Simon Peyton-Jones wrote:
> I believe fibon/ was helpfully added by someone, but never integrated
> into the nofib build system.  Just needs doing, I think

Right - I think it was even integrated into the build system, but it 
wasn't turned on by default.  I tried it once and something didn't work, 
and I didn't have the time to fix it then.

There are some other collections of programs in nofib that aren't run by 
default:

nofib/gc

My GC benchmarks (some of these overlap with the rest of nofib, but 
might have different inputs/parameters).  I usually run these when I 
change something in the GC.

nofib/smp

The concurrency benchmarks.  Edward is using these to tune his new 
scheduler.  These could be enabled by default.

nofib/parallel

The parallel benchmarks.  It wouldn't hurt to run these by default too, 
on at least 1 core and maybe more.  I generally run them on 8 cores when 
I change something in the RTS.

Cheers,
	Simon


> Simon
>
> *From:*ghc-devs-bounces at haskell.org
> [mailto:ghc-devs-bounces at haskell.org] *On Behalf Of *Nicolas Frisby
> *Sent:* 05 February 2013 09:24
> *To:* Johan Tibell
> *Cc:* ghc-devs at haskell.org
> *Subject:* Re: nofib comparisons between 7.0.4, 7.4.2, 7.6.1, and 7.6.2
>
> Is anyone familiar with the "fibon" directory within the nofib.git
> repository?
>
> http://darcs.haskell.org/nofib/fibon/
>
> Johan, this at least seems like an potential home for the additional
> programs you suggested adding. In particular, it has Repa, Dph,
> Shootout, and Hackage subdirectories.
>
> I'm doing a GHC HQ internship at the moment, and one of
> the just-needs-to-happen tasks on my (growing) todo list is to look into
> fibon.
>
> SPJ recalls that not all of the various building infrastructures were
> getting along. Anyone know the story? Thanks!
>
> On Mon, Feb 4, 2013 at 10:33 PM, Johan Tibell <johan.tibell at gmail.com
> <mailto:johan.tibell at gmail.com>> wrote:
>
> Hi all,
>
> I haven't had much time to do performance tzar work yet, but I did run
> nofib on the last few GHC releases to see the current trend. The
> benchmarks where run on my 64-bit Core i7-3770 @ 3.40GHz Linux machine.
> Here are the results:
>
> 7.0.4 to 7.4.2:
>
> --------------------------------------------------------------------------------
>          Program           Size    Allocs   Runtime   Elapsed  TotalMem
>
> --------------------------------------------------------------------------------
>
>              Min          -1.6%    -57.3%    -39.1%    -36.4%    -25.0%
>
>              Max         +21.5%   +121.5%    +24.5%    +25.4%   +300.0%
>
>   Geometric Mean          +8.5%     -0.7%     -7.1%     -5.2%     +2.0%
>
> The big loser here in terms of runtime is "kahan", which I added to test
> tight loops involving unboxed arrays and floating point arithmetic. I
> believe there was a regression in fromIntegral RULES during this
> release, which meant that some conversions between fixed-width types
> went via Integer, causing unnecessary allocation.
>
> 7.4.2 to 7.6.1:
>
> --------------------------------------------------------------------------------
>          Program           Size    Allocs   Runtime   Elapsed  TotalMem
> --------------------------------------------------------------------------------
>              Min          -5.1%    -23.8%    -11.8%    -12.9%    -50.0%
>              Max          +5.3%   +225.5%     +7.2%     +8.8%   +200.0%
>   Geometric Mean          -0.4%     +2.1%     +0.3%     +0.2%     +0.7%
>
> The biggest loser here in terms of runtime is "integrate". I haven't
> looked into why yet.
>
> 7.6.1 to 7.6.2:
>
> --------------------------------------------------------------------------------
>          Program           Size    Allocs   Runtime   Elapsed  TotalMem
>
> --------------------------------------------------------------------------------
>
>              Min          -2.9%     +0.0%     -4.8%     -4.4%     -1.9%
>
>              Max          +0.0%     +1.0%     +4.5%     +6.4%    +20.8%
>
>   Geometric Mean          -1.7%     +0.0%     +0.1%     +0.3%     +0.2%
>
> I have two takeaways:
>
>   * It's worthwhile running nofib before releases as it does find some
> programs that regressed.
>
>   * There are some other regressions out there (i.e. in code on Hackage)
> that aren't reflected here, suggesting that we need to add more programs
> to nofib.
>
> Cheers,
>
> Johan
>
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org <mailto:ghc-devs at haskell.org>
> http://www.haskell.org/mailman/listinfo/ghc-devs
>
>
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://www.haskell.org/mailman/listinfo/ghc-devs
>