nofib regressions in HEAD since 7.6.2 release
Johan Tibell
johan.tibell at gmail.com
Fri Feb 8 18:42:02 CET 2013
Here are the numbers using -O2. Again on my 64-bit Core i7-3770 @ 3.40GHz,
running Linux.
7.6.1 vs HEAD:
--------------------------------------------------------------------------------
Program Size Allocs Runtime Elapsed TotalMem
--------------------------------------------------------------------------------
anna +0.1% -1.5% 0.08 0.08 +0.0%
ansi -1.5% -0.4% 0.00 0.00 +0.0%
atom -0.4% +0.0% +0.8% +0.8% +0.0%
awards -1.0% -0.1% 0.00 0.00 +0.0%
banner -2.7% -2.1% 0.00 0.00 +0.0%
bernouilli -0.5% -2.6% 0.13 0.13 +0.0%
binary-trees +5.6% +0.0% +4.5% +2.8% +8.0%
boyer -1.5% -0.0% 0.04 0.04 +0.0%
boyer2 -2.8% -0.0% 0.01 0.01 +0.0%
bspt -0.4% -0.1% 0.01 0.01 -33.3%
cacheprof -0.5% -6.4% +6.6% +0.0% -4.0%
calendar -1.3% -0.1% 0.00 0.00 +0.0%
cichelli -2.4% +0.0% 0.06 0.06 -5.7%
circsim -1.1% -0.1% +6.6% +3.2% +1.0%
clausify -1.4% -0.0% 0.03 0.03 +0.0%
comp_lab_zift -1.3% -0.0% 0.14 0.14 -12.5%
compress -1.3% -0.0% 0.12 0.12 +0.0%
compress2 -1.7% -0.0% 0.13 0.14 -2.1%
constraints -1.4% -0.0% +4.3% +2.6% +0.0%
cryptarithm1 -2.8% +0.0% -0.5% -3.5% +0.0%
cryptarithm2 -3.3% +1.0% 0.01 0.01 +0.0%
cse -2.7% +0.0% 0.00 0.00 +0.0%
eliza -2.6% -7.1% 0.00 0.00 +0.0%
event -1.6% -0.0% 0.10 0.10 +9.5%
exp3_8 -1.4% -0.0% 0.14 0.14 -50.0%
expert -1.8% -15.5% 0.00 0.00 +0.0%
fannkuch-redux -0.9% -0.0% -1.7% -2.1% +0.0%
fasta -1.5% +8.7% +9.2% -0.6% -50.0%
fem -2.1% -0.5% 0.03 0.03 +0.0%
fft -0.6% -0.0% 0.03 0.03 +0.0%
fft2 -3.5% -24.6% 0.04 0.04 -33.3%
fibheaps -1.3% -0.2% 0.03 0.03 +0.0%
fish -2.2% +0.0% 0.01 0.01 +0.0%
fluid +0.8% -11.9% 0.01 0.01 -50.0%
fulsom -0.0% +0.0% 0.20 0.20 -6.2%
gamteb -0.8% +0.1% 0.04 0.04 +0.0%
gcd -0.6% -0.0% 0.03 0.03 +0.0%
gen_regexps -2.7% -2.1% 0.00 0.00 +0.0%
genfft -0.9% +0.3% 0.03 0.03 +0.0%
gg -0.4% -29.0% 0.02 0.02 +0.0%
grep -0.1% -2.3% 0.00 0.00 +0.0%
hidden +0.3% -3.4% -6.4% -10.4% +0.0%
hpg -0.6% +1.3% 0.10 0.10 +0.0%
ida -0.9% +1.0% 0.06 0.06 -20.0%
infer -0.8% -0.5% 0.05 0.05 -38.5%
integer -0.5% -1.0% -1.9% -2.6% +0.0%
integrate +0.6% -36.0% 0.14 -38.2% -2.9%
k-nucleotide +7.2% +5.2% +0.5% -0.6% +46.2%
kahan -0.5% -99.7% -53.0% -52.8% +0.0%
knights -1.7% +0.4% 0.01 0.01 +0.0%
lcss -2.0% -0.7% +35.0% +22.7% -3.7%
life -2.2% -0.0% 0.16 0.16 +0.0%
lift -2.0% -0.1% 0.00 0.00 +0.0%
listcompr -2.7% +0.0% 0.06 0.06 +0.0%
listcopy -2.7% +0.0% 0.06 0.06 +0.0%
maillist -2.7% -0.9% 0.04 0.04 -2.9%
mandel +0.8% -0.0% 0.05 0.05 +0.0%
mandel2 -2.7% -0.0% 0.00 0.00 +0.0%
minimax -2.5% +0.0% 0.00 0.00 +0.0%
mkhprog -2.5% -1.1% 0.00 0.00 +0.0%
multiplier -1.0% -0.0% 0.08 0.08 +0.0%
n-body +5.4% +0.0% -22.3% -23.1% +0.0%
nucleic2 +2.2% -16.3% 0.05 0.05 +0.0%
para -1.7% -18.5% -8.1% -12.8% +0.0%
paraffins -1.4% -0.0% 0.07 0.07 -7.0%
parser -1.3% -18.4% 0.03 0.03 +0.0%
parstof -1.8% -4.5% 0.00 0.00 +0.0%
pic -1.0% -0.0% 0.00 0.00 +0.0%
pidigits -0.6% -0.0% +4.8% -2.3% +0.0%
power +0.0% -0.0% +2.9% +0.5% +0.0%
pretty -1.1% -0.0% 0.00 0.00 +0.0%
primes -1.0% +0.0% 0.05 0.05 +0.0%
primetest -0.6% -0.0% 0.07 0.07 +0.0%
prolog -2.1% -14.0% 0.00 0.00 +0.0%
puzzle -2.6% +2.2% 0.10 0.10 +0.0%
queens -1.5% -0.0% 0.02 0.02 +0.0%
reptile -0.8% -0.8% 0.02 0.02 +0.0%
reverse-complem -2.5% +0.0% 0.08 0.08 +0.0%
rewrite -1.2% -0.7% 0.02 0.02 +0.0%
rfib +1.0% -0.4% 0.02 0.02 +0.0%
rsa -0.7% -2.5% 0.03 0.03 +0.0%
scc -2.8% -0.0% 0.00 0.00 +0.0%
sched -1.5% -0.0% 0.02 0.02 +0.0%
scs +2.0% -0.6% +2.2% +0.5% +0.0%
simple -0.7% +0.0% 0.16 0.16 -6.5%
solid -0.5% -0.0% 0.10 0.10 +0.0%
sorting -2.7% -35.9% 0.00 0.00 +0.0%
spectral-norm +5.4% +16.1% +0.0% +0.0% +0.0%
sphere -1.1% +1.8% 0.03 0.03 +0.0%
symalg +0.8% -0.3% 0.01 0.01 +0.0%
tak -1.5% -0.9% 0.01 0.01 +0.0%
transform -1.0% -0.0% +4.3% +2.6% +0.0%
treejoin -2.7% -52.4% 0.15 0.15 +3.8%
typecheck -1.3% -0.0% 0.15 0.15 +0.0%
veritas -0.3% -0.0% 0.00 0.00 +0.0%
wang -0.8% -0.0% 0.08 0.08 +0.0%
wave4main -1.3% -1.8% 0.19 0.19 +7.7%
wheel-sieve1 -1.0% -0.0% +8.3% +3.1% +14.3%
wheel-sieve2 -1.0% -0.0% 0.12 0.12 -2.1%
x2n1 -7.1% -48.0% 0.01 0.01 -66.7%
--------------------------------------------------------------------------------
Min -7.1% -95.0% -53.0% -52.8% -66.7%
Max +7.2% +16.1% +35.0% +22.7% +46.2%
Geometric Mean -1.0% -6.8% -1.7% -6.6% -4.4%
lcss is definitely worth looking into, as it regressed ~35%. fast might be
worth looking into, as it regressed with ~10%. There are also some ~5%
regressions that are probably not worth looking into, unless someone has
spare time.
7.6.2 vs HEAD:
--------------------------------------------------------------------------------
Program Size Allocs Runtime Elapsed TotalMem
--------------------------------------------------------------------------------
anna +0.8% -1.5% 0.08 0.08 +0.0%
ansi +0.5% -0.4% 0.00 0.00 +0.0%
atom +0.9% +0.0% +0.8% +0.8% +0.0%
awards +0.9% -0.1% 0.00 0.00 +0.0%
banner -0.1% -2.1% 0.00 0.00 +0.0%
bernouilli +0.6% -2.6% 0.13 0.13 +0.0%
binary-trees +6.9% +0.0% +3.2% +1.6% +8.0%
boyer +0.7% -0.0% 0.04 0.04 +0.0%
boyer2 -0.2% -0.0% 0.01 0.01 +0.0%
bspt +0.7% -0.1% 0.01 0.01 -33.3%
cacheprof +0.9% -6.4% +4.0% +0.0% -4.0%
calendar +0.9% -0.1% 0.00 0.00 +0.0%
cichelli +0.2% +0.0% 0.06 0.06 -5.7%
circsim +0.8% -0.1% +7.0% +1.9% +1.0%
clausify +0.8% -0.0% 0.03 0.03 +0.0%
comp_lab_zift +0.8% -0.0% 0.14 0.14 -12.5%
compress +0.1% -0.0% 0.12 0.12 +0.0%
compress2 +0.1% -0.0% 0.13 0.14 -2.1%
constraints +0.8% -0.0% +2.3% +0.2% +0.0%
cryptarithm1 +0.1% +0.0% -2.5% -4.0% +0.0%
cryptarithm2 -0.6% +1.0% 0.01 0.01 +0.0%
cse +0.0% +0.0% 0.00 0.00 +0.0%
eliza -0.1% -7.1% 0.00 0.00 +0.0%
event +0.7% -0.0% 0.10 0.10 +9.5%
exp3_8 +0.8% -0.0% 0.14 0.14 -50.0%
expert +0.2% -15.5% 0.00 0.00 +0.0%
fannkuch-redux +0.8% -0.0% +3.7% +3.7% +0.0%
fasta +0.8% -0.0% +7.8% +0.6% +0.0%
fem -1.4% -0.5% 0.03 0.03 +0.0%
fft +0.7% -0.0% 0.03 0.03 +0.0%
fft2 -3.5% -24.6% 0.04 0.04 -33.3%
fibheaps +0.9% -0.2% 0.03 0.03 +0.0%
fish +0.0% +0.0% 0.01 0.01 +0.0%
fluid +1.2% -11.9% 0.01 0.01 -50.0%
fulsom +0.2% +0.0% 0.20 0.20 -5.1%
gamteb +0.3% +0.1% 0.04 0.04 +0.0%
gcd +0.8% -0.0% 0.03 0.03 +0.0%
gen_regexps -0.0% -2.1% 0.00 0.00 +0.0%
genfft +0.8% +0.3% 0.03 0.03 +0.0%
gg +0.6% -29.0% 0.02 0.02 +0.0%
grep +2.5% -2.3% 0.00 0.00 +0.0%
hidden +0.7% -3.4% -6.4% -8.2% +0.0%
hpg +0.5% +1.3% 0.10 0.10 +0.0%
ida +0.8% +1.0% 0.06 0.06 -20.0%
infer +1.2% -0.5% 0.05 0.05 -38.5%
integer +0.9% -1.0% -4.7% -5.6% +0.0%
integrate +0.9% -36.0% -34.6% -41.9% -1.0%
k-nucleotide +8.5% +5.2% +2.7% +1.7% +46.2%
kahan +0.9% -99.7% -52.2% -52.0% +0.0%
knights +0.2% +0.4% 0.01 0.01 +0.0%
lcss +0.2% -0.7% 0.27 +25.0% -3.7%
life +0.1% -0.0% 0.16 0.16 +0.0%
lift +0.0% -0.1% 0.00 0.00 +0.0%
listcompr +0.1% +0.0% 0.06 0.06 +0.0%
listcopy +0.1% +0.0% 0.06 0.06 +0.0%
maillist -0.1% -1.8% 0.04 0.04 +6.3%
mandel +0.8% -0.0% 0.05 0.05 +0.0%
mandel2 +0.1% -0.0% 0.00 0.00 +0.0%
minimax +0.2% +0.0% 0.00 0.00 +0.0%
mkhprog +0.2% -1.1% 0.00 0.00 +0.0%
multiplier +1.2% -0.0% 0.08 0.08 +0.0%
n-body +6.6% +0.0% -21.4% -22.2% +0.0%
nucleic2 +3.4% -16.3% 0.05 0.05 +0.0%
para +0.3% -18.5% -12.8% -15.0% +0.0%
paraffins +0.8% -0.0% 0.07 0.07 -7.0%
parser +0.7% -18.4% 0.03 0.03 +0.0%
parstof +0.5% -4.5% 0.00 0.00 +0.0%
pic -0.8% -0.0% 0.00 0.00 +0.0%
pidigits +0.8% -0.0% +3.6% -0.6% +0.0%
power +1.0% -0.0% +1.4% -0.9% +0.0%
pretty +0.2% -0.0% 0.00 0.00 +0.0%
primes +0.8% +0.0% 0.05 0.05 +0.0%
primetest +0.7% -0.0% 0.07 0.07 +0.0%
prolog +0.2% -14.0% 0.00 0.00 +0.0%
puzzle +0.1% +2.2% 0.10 0.10 +0.0%
queens +0.8% -0.0% 0.02 0.02 +0.0%
reptile +0.6% -0.8% 0.02 0.02 +0.0%
reverse-complem +0.4% +0.0% 0.08 0.08 +0.0%
rewrite +1.0% -0.7% 0.02 0.02 +0.0%
rfib +1.0% -0.4% 0.02 0.02 +0.0%
rsa +0.7% -2.5% 0.03 0.03 +0.0%
scc +0.1% -0.0% 0.00 0.00 +0.0%
sched +0.8% -0.0% 0.02 0.02 +0.0%
scs +2.0% -0.6% +5.2% +1.7% +0.0%
simple +0.2% +0.0% 0.16 0.16 -6.5%
solid +0.7% -0.0% 0.10 0.10 +0.0%
sorting -0.0% -35.9% 0.00 0.00 +0.0%
spectral-norm +6.7% +16.1% +0.2% +0.0% +0.0%
sphere +0.8% +1.8% 0.03 0.03 +0.0%
symalg +0.8% -0.3% 0.01 0.01 +0.0%
tak +0.8% -0.9% 0.01 0.01 +0.0%
transform +0.9% -0.0% +6.2% +3.4% +0.0%
treejoin -0.0% -52.4% 0.15 0.15 +3.8%
typecheck +0.8% -0.0% 0.15 0.15 +0.0%
veritas +0.7% -0.0% 0.00 0.00 +0.0%
wang +0.5% -0.0% 0.08 0.08 +0.0%
wave4main +0.8% -1.8% 0.19 0.19 +7.7%
wheel-sieve1 +0.7% -0.0% +3.1% +0.8% +14.3%
wheel-sieve2 +0.7% -0.0% 0.12 0.12 -2.1%
x2n1 -6.8% -48.0% 0.01 0.01 -66.7%
--------------------------------------------------------------------------------
Min -6.8% -95.0% -52.2% -52.0% -66.7%
Max +8.5% +16.1% +7.8% +25.0% +46.2%
Geometric Mean +0.7% -6.9% -5.5% -6.6% -3.6%
7.6.1 vs 7.6.2:
--------------------------------------------------------------------------------
Program Size Allocs Runtime Elapsed TotalMem
--------------------------------------------------------------------------------
anna -0.6% +0.0% 0.07 0.08 +0.0%
ansi -2.0% +0.0% 0.00 0.00 +0.0%
atom -1.3% +0.0% +0.0% +0.0% +0.0%
awards -1.8% +0.0% 0.00 0.00 +0.0%
banner -2.6% +0.0% 0.00 0.00 +0.0%
bernouilli -1.0% +0.0% 0.13 0.13 +0.0%
binary-trees -1.2% +0.0% +1.2% +1.2% +0.0%
boyer -2.2% +0.0% 0.03 0.04 +0.0%
boyer2 -2.6% +0.0% 0.01 0.01 +0.0%
bspt -1.1% +0.0% 0.01 0.02 +0.0%
cacheprof -1.4% +0.0% +2.5% +0.0% +0.0%
calendar -2.2% +0.0% 0.00 0.00 +0.0%
cichelli -2.6% +0.0% 0.05 0.06 +0.0%
circsim -1.9% +0.0% -0.3% +1.3% +0.0%
clausify -2.2% +0.0% 0.03 0.03 +0.0%
comp_lab_zift -2.1% +0.0% 0.13 0.14 +0.0%
compress -1.4% +0.0% 0.11 0.11 +0.0%
compress2 -1.8% +0.0% 0.12 0.14 +0.0%
constraints -2.2% +0.0% +2.0% +2.4% +0.0%
cryptarithm1 -2.8% +0.0% +2.1% +0.5% +0.0%
cryptarithm2 -2.7% +0.0% 0.01 0.01 +0.0%
cse -2.7% +0.0% 0.00 0.00 +0.0%
eliza -2.5% +0.0% 0.00 0.00 +0.0%
event -2.2% +0.0% 0.09 0.10 +0.0%
exp3_8 -2.2% +0.0% 0.14 0.14 +0.0%
expert -2.0% +0.0% 0.00 0.00 +0.0%
fannkuch-redux -1.7% +0.0% -5.2% -5.6% +0.0%
fasta -2.3% +8.7% +1.3% -1.2% -50.0%
fem -0.7% +0.0% 0.02 0.03 +0.0%
fft -1.3% +0.0% 0.02 0.03 +0.0%
fft2 +0.0% +0.0% 0.04 0.05 +0.0%
fibheaps -2.2% +0.0% 0.03 0.03 +0.0%
fish -2.2% +0.0% 0.01 0.01 +0.0%
fluid -0.3% +0.0% 0.01 0.01 +0.0%
fulsom -0.2% +0.0% 0.19 0.19 -1.3%
gamteb -1.1% +0.0% 0.03 0.04 +0.0%
gcd -1.4% +0.0% 0.02 0.03 +0.0%
gen_regexps -2.7% +0.0% 0.00 0.00 +0.0%
genfft -1.7% +0.0% 0.03 0.03 +0.0%
gg -1.0% +0.0% 0.01 0.02 +0.0%
grep -2.5% +0.0% 0.00 0.00 +0.0%
hidden -0.4% +0.0% +0.0% -2.5% +0.0%
hpg -1.1% +0.0% 0.05 0.10 +0.0%
ida -1.7% +0.0% 0.06 0.06 +0.0%
infer -2.0% +0.0% 0.05 0.05 +0.0%
integer -1.3% +0.0% +3.0% +3.1% +0.0%
integrate -0.3% +0.0% 0.21 +6.4% -1.9%
k-nucleotide -1.1% +0.0% -2.2% -2.3% +0.0%
kahan -1.4% +0.0% -1.7% -1.7% +0.0%
knights -1.9% +0.0% 0.00 0.01 +0.0%
lcss -2.2% +0.0% -1.0% -1.8% +0.0%
life -2.3% +0.0% 0.16 0.16 +0.0%
lift -2.0% +0.0% 0.00 0.00 +0.0%
listcompr -2.8% +0.0% 0.06 0.06 +0.0%
listcopy -2.8% +0.0% 0.06 0.06 +0.0%
maillist -2.6% +1.0% 0.03 0.05 -8.7%
mandel -0.0% +0.0% 0.04 0.05 +0.0%
mandel2 -2.8% +0.0% 0.00 0.00 +0.0%
minimax -2.6% +0.0% 0.00 0.00 +0.0%
mkhprog -2.6% +0.0% 0.00 0.00 +0.0%
multiplier -2.2% +0.0% 0.08 0.08 +0.0%
n-body -1.2% +0.0% -1.1% -1.3% +0.0%
nucleic2 -1.2% +0.0% 0.05 0.05 +0.0%
para -2.0% +0.0% +5.4% +2.6% +0.0%
paraffins -2.2% +0.0% 0.06 0.08 +0.0%
parser -2.1% +0.0% 0.03 0.03 +0.0%
parstof -2.2% +0.0% 0.01 0.00 +0.0%
pic -0.3% +0.0% 0.00 0.00 +0.0%
pidigits -1.5% +0.0% +1.2% -1.7% +0.0%
power -1.0% +0.0% +1.4% +1.4% +0.0%
pretty -1.4% +0.0% 0.00 0.00 +0.0%
primes -1.8% +0.0% 0.05 0.05 +0.0%
primetest -1.3% +0.0% 0.08 0.07 +0.0%
prolog -2.4% +0.0% 0.00 0.00 +0.0%
puzzle -2.7% +0.0% 0.10 0.10 +0.0%
queens -2.3% +0.0% 0.02 0.02 +0.0%
reptile -1.4% +0.0% 0.01 0.02 +0.0%
reverse-complem -2.9% +0.0% 0.06 0.08 +0.0%
rewrite -2.1% +0.0% 0.02 0.02 +0.0%
rfib +0.0% +0.0% 0.02 0.02 +0.0%
rsa -1.4% +0.0% 0.02 0.02 +0.0%
scc -2.9% +0.0% 0.00 0.00 +0.0%
sched -2.2% +0.0% 0.02 0.02 +0.0%
scs +0.0% +0.0% -2.8% -1.1% +0.0%
simple -1.0% +0.0% 0.15 0.16 +0.0%
solid -1.3% +0.0% 0.09 0.10 +0.0%
sorting -2.6% +0.0% 0.00 0.00 +0.0%
spectral-norm -1.2% +0.0% -0.2% +0.0% +0.0%
sphere -1.9% +0.0% 0.03 0.03 +0.0%
symalg -0.0% +0.0% 0.00 0.01 +0.0%
tak -2.3% +0.0% 0.01 0.01 +0.0%
transform -1.9% +0.0% -1.7% -0.9% +0.0%
treejoin -2.7% +0.0% 0.15 0.16 +0.0%
typecheck -2.2% +0.0% 0.15 0.15 +0.0%
veritas -1.1% +0.0% 0.00 0.00 +0.0%
wang -1.3% +0.0% 0.07 0.08 +0.0%
wave4main -2.1% +0.0% 0.19 0.18 +0.0%
wheel-sieve1 -1.7% +0.0% +5.0% +2.4% +0.0%
wheel-sieve2 -1.7% +0.0% 0.11 0.13 +0.0%
x2n1 -0.3% +0.0% 0.01 0.01 +0.0%
--------------------------------------------------------------------------------
Min -2.9% +0.0% -5.2% -5.6% -50.0%
Max +0.0% +8.7% +5.4% +6.4% +0.0%
Geometric Mean -1.7% +0.1% +0.4% +0.0% -0.8%
The big regression Bryan and I saw in hashable between 7.6.1 and 7.6.2
isn't visible here, so we might need to extract a separate benchmark for it.
-- Johan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/ghc-devs/attachments/20130208/f2a52258/attachment-0001.htm>
More information about the ghc-devs
mailing list