On CI

[Merijn Verstraaten]

On 17 Mar 2021, at 16:16, Andreas Klebinger <klebinger.andreas at gmx.at> wrote:
> 
> While I fully agree with this. We should *always* want to know if a small syntetic benchmark regresses by a lot.
> Or in other words we don't want CI to accept such a regression for us ever, but the developer of a patch should need to explicitly ok it.
> 
> Otherwise we just slow down a lot of seldom-used code paths by a lot.
> 
> Now that isn't really an issue anyway I think. The question is rather is 2% a large enough regression to worry about? 5%? 10%?

You probably want a sliding window anyway. Having N 1.8% regressions in a row can still slow things down a lot. While a 3% regression after a 5% improvement is probably fine.

- Merijn
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: Message signed with OpenPGP
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210317/ea836587/attachment.sig>