RTS changes affect runtime when they shouldn’t

[Ben Gamari]

Joachim Breitner <mail at joachim-breitner.de> writes:

[snip]

>
> Does anyone have a solid idea what is causing these differences? Are
> they specific to the builder for perf.haskell.org, or do you observe
> them as well? And what can we do here?
>
There is certainly no shortage of possible causes:

    https://www.youtube.com/watch?v=IX16gcX4vDQ

It would be interesting to take a few days to really try to build an
understanding of a few of these performance jumps with perf. At the
moment we can only speculate.

> For the measurements in my thesis I switched to measuring instruction
> counts (using valgrind) instead. These are much more stable, requires
> only a single NoFibRun, and the machine does not have to be otherwise
> quiet. Should I start using these on perf.haskell.org? Or would we lose
> too much by not tracking actual running times any more?
>
Note that valgrind can also do cache modelling so I suspect it can give
you a reasonably good picture of execution; certainly better than
runtime. However, the trade-off is that (last I checked) it's incredibly
slow. Don't you think I mean just a bit less peppy than usual. I mean
soul-crushingly, mollasses-on-a-cold-December-morning slow.

If we think that we can bear the cost of running valigrind then I think
it would be a great improvement. As you point out, the current run times
are essentially useless.

Cheers,

- Ben
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20170920/b76242a7/attachment.sig>