nofib comparisons between 7.0.4, 7.4.2, 7.6.1, and 7.6.2

[Tim Watson]

We have some benchmarks for Cloud Haskell and its underlying network-transport
infrastructure that I'm in the process of trying to automate. I'd be very
interested to see how these fare against various GHC releases, though I suspect
we'll have to tweak the dependencies considerably in order to make the
automation happen.

I don't know if that fits into the 'other regressions' category or not?

Cheers,
Tim

On 5 Feb 2013, at 14:24, Nicolas Frisby wrote:

> I'd like to investigate the "other regressions out there".
>  
> Do you have more info? Perhaps a list? Maybe even benchmarking code?
>  
> Thanks.
> 
> 
> On Tue, Feb 5, 2013 at 4:22 AM, Austin Seipp <mad.one at gmail.com> wrote:
> I'm +1 for this. Eyal Lotem and I were just discussing this on IRC a
> few minutes ago, and he suffered a rather large (~25%) performance hit
> when upgrading to 7.6.1, which is unfortunate.
> 
> Committers are typically very good about recording nofib results in
> their commit and being performance-courteous, but I'm not sure there's
> ever been a longer-scale view of GHC performance over multiple
> releases like this - or even a few months. At least not recently. On
> top of that, his application was a type checker, which may certainly
> stress different performance points than what nofib might. Once we get
> performance bots set up, I've got a small set of machines I'm willing
> to throw at it.
> 
> Thanks for the results, Johan!
> 
> On Mon, Feb 4, 2013 at 4:33 PM, Johan Tibell <johan.tibell at gmail.com> wrote:
> > Hi all,
> >
> > I haven't had much time to do performance tzar work yet, but I did run nofib
> > on the last few GHC releases to see the current trend. The benchmarks where
> > run on my 64-bit Core i7-3770 @ 3.40GHz Linux machine. Here are the results:
> >
> > 7.0.4 to 7.4.2:
> >
> > --------------------------------------------------------------------------------
> >         Program           Size    Allocs   Runtime   Elapsed  TotalMem
> > --------------------------------------------------------------------------------
> >             Min          -1.6%    -57.3%    -39.1%    -36.4%    -25.0%
> >             Max         +21.5%   +121.5%    +24.5%    +25.4%   +300.0%
> >  Geometric Mean          +8.5%     -0.7%     -7.1%     -5.2%     +2.0%
> >
> > The big loser here in terms of runtime is "kahan", which I added to test
> > tight loops involving unboxed arrays and floating point arithmetic. I
> > believe there was a regression in fromIntegral RULES during this release,
> > which meant that some conversions between fixed-width types went via
> > Integer, causing unnecessary allocation.
> >
> > 7.4.2 to 7.6.1:
> >
> > --------------------------------------------------------------------------------
> >         Program           Size    Allocs   Runtime   Elapsed  TotalMem
> > --------------------------------------------------------------------------------
> >             Min          -5.1%    -23.8%    -11.8%    -12.9%    -50.0%
> >             Max          +5.3%   +225.5%     +7.2%     +8.8%   +200.0%
> >  Geometric Mean          -0.4%     +2.1%     +0.3%     +0.2%     +0.7%
> >
> > The biggest loser here in terms of runtime is "integrate". I haven't looked
> > into why yet.
> >
> > 7.6.1 to 7.6.2:
> >
> > --------------------------------------------------------------------------------
> >         Program           Size    Allocs   Runtime   Elapsed  TotalMem
> > --------------------------------------------------------------------------------
> >             Min          -2.9%     +0.0%     -4.8%     -4.4%     -1.9%
> >             Max          +0.0%     +1.0%     +4.5%     +6.4%    +20.8%
> >  Geometric Mean          -1.7%     +0.0%     +0.1%     +0.3%     +0.2%
> >
> > I have two takeaways:
> >
> >  * It's worthwhile running nofib before releases as it does find some
> > programs that regressed.
> >  * There are some other regressions out there (i.e. in code on Hackage) that
> > aren't reflected here, suggesting that we need to add more programs to
> > nofib.
> >
> > Cheers,
> > Johan
> >
> >
> > _______________________________________________
> > ghc-devs mailing list
> > ghc-devs at haskell.org
> > http://www.haskell.org/mailman/listinfo/ghc-devs
> >
> 
> 
> 
> --
> Regards,
> Austin
> 
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://www.haskell.org/mailman/listinfo/ghc-devs
> 
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://www.haskell.org/mailman/listinfo/ghc-devs