merijn at inconsistent.nl
Wed Mar 17 16:01:53 UTC 2021
On 17 Mar 2021, at 16:16, Andreas Klebinger <klebinger.andreas at gmx.at> wrote:
> While I fully agree with this. We should *always* want to know if a small syntetic benchmark regresses by a lot.
> Or in other words we don't want CI to accept such a regression for us ever, but the developer of a patch should need to explicitly ok it.
> Otherwise we just slow down a lot of seldom-used code paths by a lot.
> Now that isn't really an issue anyway I think. The question is rather is 2% a large enough regression to worry about? 5%? 10%?
You probably want a sliding window anyway. Having N 1.8% regressions in a row can still slow things down a lot. While a 3% regression after a 5% improvement is probably fine.
-------------- next part --------------
A non-text attachment was scrubbed...
Size: 833 bytes
Desc: Message signed with OpenPGP
More information about the ghc-devs