[Haskell-cafe] Ordering of BigFloats in numbers-3000.0.0.0

[Michael Orlitzky]

On 10/09/2012 12:29 PM, Chaddaï Fouché wrote:
> On Sun, Oct 7, 2012 at 8:00 PM, Michael Orlitzky <michael at orlitzky.com> wrote:
>> I'm trying to use,
>>
>>   http://hackage.haskell.org/package/numbers-3000.0.0.0
>>
>> to get better precision "for free" out of some numerical code. I ran
>> into an issue pretty quickly, though. In Data.Number.BigFloat, we have,
>>
>>   data BigFloat e = BF (Fixed e) Integer
>>     deriving (Eq, Ord)
>>
>> and the derived Ord is obviously incorrect:
>>
>>   Prelude Data.Number.BigFloat> let x = 0.1 :: BigFloat Prec50
>>   Prelude Data.Number.BigFloat> let y = 0.02 :: BigFloat Prec50
>>   Prelude Data.Number.BigFloat> x < y
>>   True
>>
> 
> That's pretty strange since the derived Ord should be the same as
> Fixed Ord, which itself is just a newtype over Rational (and 1%10 <
> 2%100 == False), did you try to convert those BigFloat back to
> Rational to see if they're still correct ? That may be worse than a
> misbehaving Ord instance.
> 

The BigFloat constructor doesn't do what you think it does... I chose a
bad spot to snip the comments:

  -- This representation is stupid, two Integers makes more sense,
  -- but is more work.
  -- | Floating point number where the precision is determined by the
  --   type /e/.
  data BigFloat e = BF (Fixed e) Integer
    deriving (Eq)

If it was two Integers, I think it would be more clear that this is what
will happen:

  *Data.Number.BigFloat> let x = 0.1 :: BigFloat Prec50
  *Data.Number.BigFloat> let y = 0.02 :: BigFloat Prec50
  *Data.Number.BigFloat> let BF m1 e1 = x
  *Data.Number.BigFloat> let BF m2 e2 = y
  *Data.Number.BigFloat> (m1,e1)
  (1.00000000000000000000000000000000000000000000000000,-1)
  *Data.Number.BigFloat> (m2,e2)
  (2.00000000000000000000000000000000000000000000000000,-2)

The fast way to compare them would be to check the mantissa/exponent
directly, but toRational works just fine. I made a pull request
yesterday that compares them as rationals.