nofib output difference

[Ben Gamari]

Joachim Breitner <mail at joachim-breitner.de> writes:

> Hi,
>
> currently, perf.haskell.org has stopped because recent commits fail to
> build with:
>
> expected stdout not matched by reality
> --- fft2.stdout7	2018-11-28 12:00:24.126015465 -0500
> +++ /tmp/runtest6392.1	2018-11-28 12:59:59.329423367 -0500
> @@ -1,3 +1,3 @@
> -result1 = 2.59635799135966e-12
> -result2 = 2.59635799135966e-12
> -result3 = 4.8279900966008427e-8
> +result1 = 2.649891777856828e-12
> +result2 = 2.649891777856828e-12
> +result3 = 4.830900479646516e-8
> make[2]: *** [../../mk/target.mk:102: runtests] Error 1
>
> do others observe this as well? This is worrying, because there has
> been no change to nofib since it last worked.
>
> Can we add a check of the functional correctness of the nofib programs
> to the CI somehow?
>
Yikes, this is indeed somewhat worrying. I see the same result as you
for result1 and result2 but a slightly different number for result3.

Looking at this particular test it appears to be quite numerically
unstable. I count over ten distinct sets of .stdout files in
spectral/fft2, some of which match what the results I observe quite
closely.

Overall we should indeed make sure these get tested in CI.

Do you know how long ago this change occurred?

Cheers,

- Ben
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20181129/ca56338a/attachment.sig>