nofib regressions in HEAD since 7.6.2 release

Nicolas Frisby nicolas.frisby at gmail.com
Fri Feb 8 17:31:18 CET 2013


On Fri, Feb 8, 2013 at 1:28 PM, Simon Marlow <marlowsd at gmail.com> wrote:

> On 08/02/13 10:10, Nicolas Frisby wrote:
>
>> Hi Johan. I'm seeing much different numbers when I test for regressions.
>> Also 64-bit Linux.
>> Could you explain your method? Origin of the GHC/libraries you're
>> measuring and the options your running NoFib with? Thanks.
>> I'm using the 28 Jan ghc-7.6.2-x86_64-unknown-**linux.tar.bz2 and
>> ec9377b1e51fd59e18 (Wed Feb 6 14:10:39 2013 +0000) for HEAD.
>> Thanks.
>>
>
> The usual source of differences is -O vs. -O2, either for the benchmarks
> or for the libraries: you want GhcLibOpts=-O2 and NoFibHcOpts=-O2 in your
> mk/build.mk file.
>
> Cheers,
>         Simon
>
>
I figured Johan is doing O2 for libraries and programs. I have done
libraries at O2, programs at O1, which likely does explain the difference.

I'm still in a holding pattern in case there's also some other setting he's
using.

Then I'll run both variations on the programs.

Thanks much.



>
>
>
>> On Thu, Feb 7, 2013 at 9:18 PM, Johan Tibell <johan.tibell at gmail.com
>> <mailto:johan.tibell at gmail.com**>> wrote:
>>
>>     Hi all,
>>
>>     I just ran nofib on current HEAD and compared it to 7.6.2 on my
>>     64-bit Linux machine. There are some regressions I think we should
>>     look into before a release:
>>
>>     ------------------------------**------------------------------**
>> --------------------
>>              Program           Size    Allocs   Runtime   Elapsed
>>  TotalMem
>>     ------------------------------**------------------------------**
>> --------------------
>>                 anna          -0.8%     +1.5%      0.08      0.08
>> +0.0%
>>                 ansi          -0.5%     +0.5%      0.00      0.00
>> +0.0%
>>                 atom          -0.9%     -0.0%     -3.1%     -0.8%
>> +0.0%
>>               awards          -0.9%     +0.1%      0.00      0.00
>> +0.0%
>>               banner          +0.1%     +2.2%      0.00      0.00
>> +0.0%
>>           bernouilli          -0.6%     +2.7%      0.12      0.13
>> +0.0%
>>                boyer          -0.7%     +0.0%      0.03      0.04
>> +0.0%
>>               boyer2          -0.1%     +0.0%      0.01      0.01
>> +0.0%
>>                 bspt          -0.6%     -0.0%      0.01      0.02
>> +0.0%
>>            cacheprof          -0.9%     +6.9%     +0.8%     +4.0%
>> +0.0%
>>             calendar          -0.9%     +0.1%      0.00      0.00
>> +0.0%
>>             cichelli          -0.1%     -0.0%      0.06      0.06
>> +9.4%
>>              circsim          -0.7%     +0.1%     +1.6%     +7.0%
>>  +13.6%
>>             clausify          -0.8%     +0.0%      0.03      0.03
>> +0.0%
>>        comp_lab_zift          -0.7%     +0.0%      0.14      0.14
>>  -12.5%
>>             compress          -0.1%     +0.0%      0.12      0.13
>> +0.0%
>>            compress2          -0.0%     +0.0%      0.12      0.14
>> +2.3%
>>          constraints          -0.8%     +0.0%     -3.3%     -0.7%
>> +0.0%
>>         cryptarithm1          -0.1%     +0.0%     +2.1%     +4.1%
>> +0.0%
>>         cryptarithm2          -0.0%     -0.8%      0.01      0.01
>> +0.0%
>>                  cse          -0.0%     -0.0%      0.00      0.00
>> +0.0%
>>                eliza          +0.1%     +7.7%      0.00      0.00
>> +0.0%
>>                event          -0.8%     +0.0%      0.09      0.10
>> -8.7%
>>               exp3_8          -0.8%     +0.0%      0.15      0.15
>> +100.0%
>>               expert          -0.2%    +18.4%      0.00      0.00
>> +0.0%
>>                  fem          +1.5%     +0.5%      0.02      0.02
>> +0.0%
>>                  fft          -0.9%     +0.0%      0.02      0.03
>> +0.0%
>>                 fft2          +3.7%    +31.9%      0.05      0.05
>>  +20.0%
>>             fibheaps          -1.0%     +0.3%      0.03      0.03
>> +0.0%
>>                 fish          -0.0%     -0.0%      0.01      0.01
>> +0.0%
>>                fluid          -1.3%    +13.0%      0.01      0.01
>> +0.0%
>>               fulsom          -0.2%     -0.0%      0.19      0.20
>> +8.3%
>>               gamteb          -0.7%     -0.2%      0.03      0.04
>> +0.0%
>>                  gcd          -0.8%     +0.0%      0.02      0.03
>> +0.0%
>>          gen_regexps          +0.0%     +2.2%      0.00      0.00
>> +0.0%
>>               genfft          -0.8%     -0.3%      0.03      0.03
>> +0.0%
>>                   gg          -0.6%    +40.4%      0.01      0.02
>>  +50.0%
>>                 grep          -1.8%     +2.3%      0.00      0.00
>> +0.0%
>>               hidden          -0.7%     +3.5%     +4.1%     +7.6%
>> +0.0%
>>                  hpg          -1.3%     -1.3%      0.05      0.10
>> +0.0%
>>                  ida          -0.7%     -1.0%      0.07      0.08
>>  +11.1%
>>                infer          -0.9%     +0.5%      0.05      0.05
>>  +30.0%
>>              integer          -0.8%     +1.1%     +1.1%     +2.3%
>> +0.0%
>>            integrate          -0.9%    +56.2%      0.20      0.23
>> +1.0%
>>                kahan          -0.9%   +144.9%    +77.6%    +78.4%
>> +0.0%
>>              knights          -0.1%     -0.4%      0.01      0.01
>> +0.0%
>>                 lcss          -0.7%     +0.7%    -24.3%    -18.4%
>> +1.9%
>>                 life          -0.2%     +0.0%      0.16      0.16
>> +0.0%
>>                 lift          -0.0%     +0.1%      0.00      0.00
>> +0.0%
>>            listcompr          -0.1%     -0.0%      0.06      0.06
>> +0.0%
>>             listcopy          -0.1%     -0.0%      0.06      0.06
>> +0.0%
>>             maillist          +0.0%     +1.9%      0.02      0.04
>>  +16.5%
>>               mandel          -0.8%     +0.0%      0.05      0.05
>> +0.0%
>>              mandel2          -0.1%     -4.0%      0.00      0.01
>> +0.0%
>>              minimax          -0.2%     -0.0%      0.00      0.00
>> +0.0%
>>              mkhprog          -0.1%     +1.1%      0.00      0.00
>> +0.0%
>>           multiplier          -1.2%     +0.0%      0.07      0.08
>> +0.0%
>>             nucleic2          -3.3%    +19.5%      0.05      0.05
>> +0.0%
>>                 para          -0.0%    +25.0%      0.22      0.23
>> +0.0%
>>            paraffins          -0.8%     +0.0%      0.06      0.08
>> +7.5%
>>               parser          -1.3%    +22.2%      0.03      0.03
>> +0.0%
>>              parstof          -0.5%     +4.6%      0.01      0.00
>> +0.0%
>>                  pic          +0.5%     +0.0%      0.00      0.00
>> +0.0%
>>                power          -1.0%     +0.0%     -0.5%     +1.0%
>> +0.0%
>>               pretty          -0.2%     +0.0%      0.00      0.00
>> +0.0%
>>               primes          -0.8%     -0.0%      0.04      0.05
>> +0.0%
>>            primetest          -0.7%     +0.0%      0.07      0.07
>> +0.0%
>>               prolog          -0.2%    +16.0%      0.00      0.00
>> +0.0%
>>               puzzle          -0.1%     -2.1%      0.09      0.10
>> +0.0%
>>               queens          -0.8%     +0.0%      0.02      0.02
>> +0.0%
>>              reptile          -0.8%     +0.6%      0.01      0.02
>> +0.0%
>>              rewrite          -0.8%     +0.7%      0.02      0.02
>> +0.0%
>>                 rfib          -1.0%     +0.4%      0.02      0.02
>> +0.0%
>>                  rsa          -0.7%     +2.5%      0.02      0.02
>> +0.0%
>>                  scc          -0.1%     +0.0%      0.00      0.00
>> +0.0%
>>                sched          -0.8%     +0.0%      0.01      0.02
>> +0.0%
>>                  scs          -1.9%     +0.8%     -5.3%     -2.7%
>> +0.0%
>>               simple          -0.3%     -0.0%      0.15      0.16
>> +6.9%
>>                solid          -0.7%     +0.0%      0.09      0.09
>> +0.0%
>>              sorting          +0.0%    +55.9%      0.00      0.00
>> +0.0%
>>               sphere          -0.9%     -1.8%      0.04      0.04
>> +0.0%
>>               symalg          -0.8%     +0.3%      0.01      0.01
>> +0.0%
>>                  tak          -0.8%     +0.9%      0.01      0.01
>> +0.0%
>>            transform          -0.8%     +0.0%     -3.3%     -5.7%
>> +0.0%
>>             treejoin          +0.1%   +109.3%      0.15      0.17
>> -7.4%
>>            typecheck          -0.8%     +0.0%      0.14      0.15
>> +0.0%
>>              veritas          -0.8%     +0.0%      0.00      0.00
>> +0.0%
>>                 wang          -0.9%     +0.0%      0.07      0.08
>> +0.0%
>>            wave4main          -0.8%     +1.8%      0.18      0.19
>> -7.1%
>>         wheel-sieve1          -0.7%     +0.0%     +0.0%     +1.5%
>>  -12.5%
>>         wheel-sieve2          -0.7%     +0.0%      0.11      0.12
>> +2.1%
>>                 x2n1          +7.4%    +43.3%      0.01      0.01
>> +200.0%
>>     ------------------------------**------------------------------**
>> --------------------
>>                  Min          -3.3%     -4.0%    -24.3%    -18.4%
>>  -12.5%
>>                  Max          +7.4%   +144.9%    +77.6%    +78.4%
>> +200.0%
>>       Geometric Mean          -0.5%     +5.5%     +1.8%     +4.3%
>> +3.2%
>>
>>     I haven't had time to look through the regressions yet, so if
>>     someone has time, please grab a benchmark that looks bad and have a
>>     look at the Core to see what's going on. I suggest starting with
>>     "kahan", which is a relatively simple benchmark.
>>
>>     -- Johan
>>
>>
>>     ______________________________**_________________
>>     ghc-devs mailing list
>>     ghc-devs at haskell.org <mailto:ghc-devs at haskell.org>
>>     http://www.haskell.org/**mailman/listinfo/ghc-devs<http://www.haskell.org/mailman/listinfo/ghc-devs>
>>
>>
>>
>>
>>
>> ______________________________**_________________
>> ghc-devs mailing list
>> ghc-devs at haskell.org
>> http://www.haskell.org/**mailman/listinfo/ghc-devs<http://www.haskell.org/mailman/listinfo/ghc-devs>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/ghc-devs/attachments/20130208/84e27fb8/attachment-0001.htm>


More information about the ghc-devs mailing list