[GHC] #8285: unexpected behavior with encodeFloat on large inputs

[GHC]

#8285: unexpected behavior with encodeFloat on large inputs
-------------------------------------+-------------------------------------
        Reporter:  carter            |                Owner:
            Type:  bug               |               Status:  new
        Priority:  normal            |            Milestone:  8.0.1
       Component:  Compiler          |              Version:  7.7
      Resolution:                    |             Keywords:
Operating System:  Unknown/Multiple  |         Architecture:
 Type of failure:  Incorrect result  |  Unknown/Multiple
  at runtime                         |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):
       Wiki Page:                    |
-------------------------------------+-------------------------------------

@@ -1,1 +1,3 @@
- benjamin scarlet discovered some unexpected behavior in encodeFloat today
+ {{{
+ > encodeFloat 1 1023
+ 8.98846567431158e307  -- ok
@@ -3,3 +5,2 @@
- {{{
- <bscarlet>   > map (\x -> encodeFloat 1 (shiftL 1 x)) [30,31]
- [23:47:47] <lambdabot>     [Infinity,0.0]
+ > encodeFloat 1 1024
+ Infinity  -- ok
@@ -7,2 +8,2 @@
- [23:49:09] <arkeet>  > encodeFloat 1 60
- [23:49:10] <lambdabot>     1.152921504606847e18
+ > encodeFloat 1 (2^31 - 1)
+ Infinity  -- ok
@@ -10,16 +11,2 @@
- [23:49:09] <arkeet>  > encodeFloat 1 60
- [23:49:10] <lambdabot>     1.152921504606847e18
- [23:49:18] <arkeet>  er.
- [23:50:21] <bscarlet>    down in the sane range I suspect it's just fine.
- [23:50:43] <bscarlet>    > encodeFloat 1 1000
- [23:50:44] <lambdabot>     1.0715086071862673e301
- [23:50:52] <bscarlet>    > encodeFloat 1 10000
- [23:50:53] <lambdabot>     Infinity
- [23:50:58] <bscarlet>    That's sane.
- [23:51:03] <arkeet>  > encodeFloat 1 (2^31)
- [23:51:04] <lambdabot>     0.0
- [23:51:13] <bscarlet>    That's what I think is weird.
- [23:51:19] <carter>  bscarlet floats are weird
- [23:51:23] <arkeet>  > encodeFloat 1 (2^31-1)
- [23:51:24] <lambdabot>     Infinity
-
+ > encodeFloat 1 (2^31)
+ 0.0  -- not ok
@@ -27,6 +14,0 @@
- Benjamin did a wee bit of investigation, and the
- problem seems to lie in https://github.com/ghc/packages-integer-
- gmp/blob/master/cbits/float.c#L177
- in integer-gmp, where it deliberately doesn't handle the edge cases
- (probably worth investigating) if integer-simple has similar behavior or
- not

New description:

 {{{
 > encodeFloat 1 1023
 8.98846567431158e307  -- ok

 > encodeFloat 1 1024
 Infinity  -- ok

 > encodeFloat 1 (2^31 - 1)
 Infinity  -- ok

 > encodeFloat 1 (2^31)
 0.0  -- not ok
 }}}

--

Comment (by thomie):

 I think this just needs an update to the `encodeFloat` docstring.

 To confirms what hvr said in comment:2:

 test.c:
 {{{
 #include <math.h>
 #include <stdio.h>
 #include <errno.h>
 #include <fenv.h>
 #include <limits.h>
 #include <float.h>

 void test(double x, int exp) {
   errno = 0;
   feclearexcept(FE_ALL_EXCEPT);
   printf("(%f)*2^(%d) = %f\n", x, exp, ldexp(x, exp));
   printf("errno: %d (ERANGE = %d), UNDERFLOW: %d, OVERFLOW: %d\n",
       errno, ERANGE,
       fetestexcept(FE_UNDERFLOW) == FE_UNDERFLOW,
       fetestexcept(FE_OVERFLOW) == FE_OVERFLOW);
 }

 void main() {
   // If the result overflows, a range error occurs, and the functions
   // return HUGE_VAL, HUGE_VALF, or HUGE_VALL, respectively, with a sign
   // the same as x.
   test(1, 1024);
   test(1, INT_MAX); // 2^31 - 1
   printf("\n");

   // If the result underflows, a range error occurs, and zero is
   // returned.
   test(1, DBL_MIN_EXP); // Not sure why not getting range error here,
   test(1, -1074); // or here.
   test(1, -1075); // but only starting here.
   test(1, 1 << 31);
   printf("\n");

   // If x is positive infinity (negative infinity), positive infinity
   // (negative infinity) is returned.
   test(HUGE_VAL, 1 << 31);
   test(- HUGE_VAL, 1 << 31);
 }
 }}}

 {{{
 $ gcc test.c -lm
 $ ./a.out
 (1.000000)*2^(1024) = inf
 errno: 34 (ERANGE = 34), UNDERFLOW: 0, OVERFLOW: 1
 (1.000000)*2^(2147483647) = inf
 errno: 34 (ERANGE = 34), UNDERFLOW: 0, OVERFLOW: 1

 (1.000000)*2^(-1021) = 0.000000
 errno: 0 (ERANGE = 34), UNDERFLOW: 0, OVERFLOW: 0
 (1.000000)*2^(-1074) = 0.000000
 errno: 0 (ERANGE = 34), UNDERFLOW: 0, OVERFLOW: 0
 (1.000000)*2^(-1075) = 0.000000
 errno: 34 (ERANGE = 34), UNDERFLOW: 1, OVERFLOW: 0
 (1.000000)*2^(-2147483648) = 0.000000
 errno: 34 (ERANGE = 34), UNDERFLOW: 1, OVERFLOW: 0

 (inf)*2^(-2147483648) = inf
 errno: 0 (ERANGE = 34), UNDERFLOW: 0, OVERFLOW: 0
 (-inf)*2^(-2147483648) = -inf
 errno: 0 (ERANGE = 34), UNDERFLOW: 0, OVERFLOW: 0
 }}}

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/8285#comment:7>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler