Nofib runtimes sensitivity to RTS changes
ben at smart-cactus.org
Sun Feb 12 15:56:44 UTC 2017
Joachim Breitner <mail at joachim-breitner.de> writes:
> this is mostly a rant, targeted at the very vague fact that runtime
> performance benchmarks are so damn hard.
> This commit just came in:
> Clearly, only parts of the profiling RTS have changed.
> This is what perf.haskell.org measures:
> nofib/time/cryptarithm1 0.502 + 9.36% 0.549 seconds
> nofib/time/fasta 0.406 + 4.43% 0.424 seconds
> These changes are permanent, i.e. the following commits consistently
> give the higher runtimes.
> But nofib does not even run the profiling RTS!
> *very audible sigh*
> (Conclusion: Ignore runtime changes in nofib when the RTS has changed,
> the results are simply unusable.)
> I also wonder if we can adjust the input and parameters to nofib (in a
> new “veryslow” setting) to get them all run in a few seconds on modern
> hardware, instead of subseconds.
I think this would be a very worthwhile effort. Sub-second runtimes
really are just plain useless.
-------------- next part --------------
A non-text attachment was scrubbed...
Size: 487 bytes
Desc: not available
More information about the ghc-devs