nofib regressions in HEAD since 7.6.2 release
Simon Marlow
marlowsd at gmail.com
Fri Feb 8 14:28:28 CET 2013
On 08/02/13 10:10, Nicolas Frisby wrote:
> Hi Johan. I'm seeing much different numbers when I test for regressions.
> Also 64-bit Linux.
> Could you explain your method? Origin of the GHC/libraries you're
> measuring and the options your running NoFib with? Thanks.
> I'm using the 28 Jan ghc-7.6.2-x86_64-unknown-linux.tar.bz2 and
> ec9377b1e51fd59e18 (Wed Feb 6 14:10:39 2013 +0000) for HEAD.
> Thanks.
The usual source of differences is -O vs. -O2, either for the benchmarks
or for the libraries: you want GhcLibOpts=-O2 and NoFibHcOpts=-O2 in
your mk/build.mk file.
Cheers,
Simon
>
> On Thu, Feb 7, 2013 at 9:18 PM, Johan Tibell <johan.tibell at gmail.com
> <mailto:johan.tibell at gmail.com>> wrote:
>
> Hi all,
>
> I just ran nofib on current HEAD and compared it to 7.6.2 on my
> 64-bit Linux machine. There are some regressions I think we should
> look into before a release:
>
> --------------------------------------------------------------------------------
> Program Size Allocs Runtime Elapsed TotalMem
> --------------------------------------------------------------------------------
> anna -0.8% +1.5% 0.08 0.08 +0.0%
> ansi -0.5% +0.5% 0.00 0.00 +0.0%
> atom -0.9% -0.0% -3.1% -0.8% +0.0%
> awards -0.9% +0.1% 0.00 0.00 +0.0%
> banner +0.1% +2.2% 0.00 0.00 +0.0%
> bernouilli -0.6% +2.7% 0.12 0.13 +0.0%
> boyer -0.7% +0.0% 0.03 0.04 +0.0%
> boyer2 -0.1% +0.0% 0.01 0.01 +0.0%
> bspt -0.6% -0.0% 0.01 0.02 +0.0%
> cacheprof -0.9% +6.9% +0.8% +4.0% +0.0%
> calendar -0.9% +0.1% 0.00 0.00 +0.0%
> cichelli -0.1% -0.0% 0.06 0.06 +9.4%
> circsim -0.7% +0.1% +1.6% +7.0% +13.6%
> clausify -0.8% +0.0% 0.03 0.03 +0.0%
> comp_lab_zift -0.7% +0.0% 0.14 0.14 -12.5%
> compress -0.1% +0.0% 0.12 0.13 +0.0%
> compress2 -0.0% +0.0% 0.12 0.14 +2.3%
> constraints -0.8% +0.0% -3.3% -0.7% +0.0%
> cryptarithm1 -0.1% +0.0% +2.1% +4.1% +0.0%
> cryptarithm2 -0.0% -0.8% 0.01 0.01 +0.0%
> cse -0.0% -0.0% 0.00 0.00 +0.0%
> eliza +0.1% +7.7% 0.00 0.00 +0.0%
> event -0.8% +0.0% 0.09 0.10 -8.7%
> exp3_8 -0.8% +0.0% 0.15 0.15 +100.0%
> expert -0.2% +18.4% 0.00 0.00 +0.0%
> fem +1.5% +0.5% 0.02 0.02 +0.0%
> fft -0.9% +0.0% 0.02 0.03 +0.0%
> fft2 +3.7% +31.9% 0.05 0.05 +20.0%
> fibheaps -1.0% +0.3% 0.03 0.03 +0.0%
> fish -0.0% -0.0% 0.01 0.01 +0.0%
> fluid -1.3% +13.0% 0.01 0.01 +0.0%
> fulsom -0.2% -0.0% 0.19 0.20 +8.3%
> gamteb -0.7% -0.2% 0.03 0.04 +0.0%
> gcd -0.8% +0.0% 0.02 0.03 +0.0%
> gen_regexps +0.0% +2.2% 0.00 0.00 +0.0%
> genfft -0.8% -0.3% 0.03 0.03 +0.0%
> gg -0.6% +40.4% 0.01 0.02 +50.0%
> grep -1.8% +2.3% 0.00 0.00 +0.0%
> hidden -0.7% +3.5% +4.1% +7.6% +0.0%
> hpg -1.3% -1.3% 0.05 0.10 +0.0%
> ida -0.7% -1.0% 0.07 0.08 +11.1%
> infer -0.9% +0.5% 0.05 0.05 +30.0%
> integer -0.8% +1.1% +1.1% +2.3% +0.0%
> integrate -0.9% +56.2% 0.20 0.23 +1.0%
> kahan -0.9% +144.9% +77.6% +78.4% +0.0%
> knights -0.1% -0.4% 0.01 0.01 +0.0%
> lcss -0.7% +0.7% -24.3% -18.4% +1.9%
> life -0.2% +0.0% 0.16 0.16 +0.0%
> lift -0.0% +0.1% 0.00 0.00 +0.0%
> listcompr -0.1% -0.0% 0.06 0.06 +0.0%
> listcopy -0.1% -0.0% 0.06 0.06 +0.0%
> maillist +0.0% +1.9% 0.02 0.04 +16.5%
> mandel -0.8% +0.0% 0.05 0.05 +0.0%
> mandel2 -0.1% -4.0% 0.00 0.01 +0.0%
> minimax -0.2% -0.0% 0.00 0.00 +0.0%
> mkhprog -0.1% +1.1% 0.00 0.00 +0.0%
> multiplier -1.2% +0.0% 0.07 0.08 +0.0%
> nucleic2 -3.3% +19.5% 0.05 0.05 +0.0%
> para -0.0% +25.0% 0.22 0.23 +0.0%
> paraffins -0.8% +0.0% 0.06 0.08 +7.5%
> parser -1.3% +22.2% 0.03 0.03 +0.0%
> parstof -0.5% +4.6% 0.01 0.00 +0.0%
> pic +0.5% +0.0% 0.00 0.00 +0.0%
> power -1.0% +0.0% -0.5% +1.0% +0.0%
> pretty -0.2% +0.0% 0.00 0.00 +0.0%
> primes -0.8% -0.0% 0.04 0.05 +0.0%
> primetest -0.7% +0.0% 0.07 0.07 +0.0%
> prolog -0.2% +16.0% 0.00 0.00 +0.0%
> puzzle -0.1% -2.1% 0.09 0.10 +0.0%
> queens -0.8% +0.0% 0.02 0.02 +0.0%
> reptile -0.8% +0.6% 0.01 0.02 +0.0%
> rewrite -0.8% +0.7% 0.02 0.02 +0.0%
> rfib -1.0% +0.4% 0.02 0.02 +0.0%
> rsa -0.7% +2.5% 0.02 0.02 +0.0%
> scc -0.1% +0.0% 0.00 0.00 +0.0%
> sched -0.8% +0.0% 0.01 0.02 +0.0%
> scs -1.9% +0.8% -5.3% -2.7% +0.0%
> simple -0.3% -0.0% 0.15 0.16 +6.9%
> solid -0.7% +0.0% 0.09 0.09 +0.0%
> sorting +0.0% +55.9% 0.00 0.00 +0.0%
> sphere -0.9% -1.8% 0.04 0.04 +0.0%
> symalg -0.8% +0.3% 0.01 0.01 +0.0%
> tak -0.8% +0.9% 0.01 0.01 +0.0%
> transform -0.8% +0.0% -3.3% -5.7% +0.0%
> treejoin +0.1% +109.3% 0.15 0.17 -7.4%
> typecheck -0.8% +0.0% 0.14 0.15 +0.0%
> veritas -0.8% +0.0% 0.00 0.00 +0.0%
> wang -0.9% +0.0% 0.07 0.08 +0.0%
> wave4main -0.8% +1.8% 0.18 0.19 -7.1%
> wheel-sieve1 -0.7% +0.0% +0.0% +1.5% -12.5%
> wheel-sieve2 -0.7% +0.0% 0.11 0.12 +2.1%
> x2n1 +7.4% +43.3% 0.01 0.01 +200.0%
> --------------------------------------------------------------------------------
> Min -3.3% -4.0% -24.3% -18.4% -12.5%
> Max +7.4% +144.9% +77.6% +78.4% +200.0%
> Geometric Mean -0.5% +5.5% +1.8% +4.3% +3.2%
>
> I haven't had time to look through the regressions yet, so if
> someone has time, please grab a benchmark that looks bad and have a
> look at the Core to see what's going on. I suggest starting with
> "kahan", which is a relatively simple benchmark.
>
> -- Johan
>
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org <mailto:ghc-devs at haskell.org>
> http://www.haskell.org/mailman/listinfo/ghc-devs
>
>
>
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://www.haskell.org/mailman/listinfo/ghc-devs
>
More information about the ghc-devs
mailing list