FLOAT PRECISSION
Tom Pledger
Tom.Pledger at peace.com
Tue Dec 16 08:52:36 EST 2003
mike V writes:
| What is the precission with which floating point operations are performed ?
| The following example shows an incorrect answer in the second case ?
| How can this problem be addressed i.e. the apparent precission seems to be a
| sevre limitation to any scientific computation ?
|
| Prelude> 1000.11 - 0.1
| 1000.01
| Prelude> 10000.11 - 0.1
| 10000.0
It seems to be a formatting thing, because:
Prelude> 10000.11 - 0.1 - 10000
0.0107422
Also beware that some builds of Hugs give you single precision floats
for the Double type. That's probably why I only got 7 decimal places
just then, instead of 15. You can get proper double precision floats
by doing your own Hugs build with the appropriate flag set.
- Tom
More information about the Hugs-Bugs
mailing list