FLOAT PRECISSION

Tom Pledger Tom.Pledger at peace.com
Tue Dec 16 08:52:36 EST 2003


mike V writes:
 | What is the precission with which floating point operations are performed ?
 | The following example shows an incorrect answer in the second case ?
 | How can this problem be addressed i.e. the apparent precission seems to be a 
 | sevre limitation to any scientific computation ?
 | 
 | Prelude> 1000.11 - 0.1
 | 1000.01
 | Prelude> 10000.11 - 0.1
 | 10000.0

It seems to be a formatting thing, because:

    Prelude> 10000.11 - 0.1 - 10000
    0.0107422

Also beware that some builds of Hugs give you single precision floats
for the Double type.  That's probably why I only got 7 decimal places
just then, instead of 15.  You can get proper double precision floats
by doing your own Hugs build with the appropriate flag set.

- Tom



More information about the Hugs-Bugs mailing list