Proposal: Add "fma" to the RealFloat class

Tue May 5 02:54:03 UTC 2015

pardon the wall of text everyone, but I really want some FMA tooling :)

I am going to spend some time later this week and next adding FMA primops
to GHC and playing around with different ways to add it to Num (which seems
pretty straightforward, though I think we'd all agree it shouldn't be
exported by Prelude). And then depending on how Yitzchak's reproposal  of
that exactly goes (or some iteration thereof) we can get something
useful/usable into 7.12

i have codes (ie *dotproducts*!!!!!) where a faster direct FMA for *exact
numbers*, and a higher precision FMA for *approximate numbers *(*ie
floating point*),  and where I cant sanely use FMA if it lives anywhere but
Num unless I rub typeable everywhere and do runtime type checks for
applicable floating point types, which kinda destroys parametrically in
engineering nice things.

@levent: ghc doesn't do any optimization for floating point arithmetic
(aside from 1-2 very simple things that are possibly questionable), and
until ghc has support for precisly emulating high precision floating point
computation in a portable way, probably wont have any interesting floating
point computation.  Mandating that fma a b c === a*b+c for inexact number
datatypes doesn't quite make sense to me. Relatedly, its a GOOD thing ghc
is conservative about optimizing floating point, because it makes doing
correct stability analyses tractable!  I look forward to the day that GHC
gets a bit more sophisticated about optimizing floating point computation,
but that day is still a ways off.

relatedly: FMA for float and double are not generally going to be faster
than the individual primitive operations, merely more accurate when used
carefully.

point being*, i'm +1 on adding some manner of FMA operations to Num* (only
sane place to put it where i can actually use it for a general use library)
and i dont really care if we name it fusedMultiplyAdd, multiplyAndAdd
accursedFusionOfSemiRingOperations, or fma. i'd favor "fusedMultiplyAdd" if
we want a descriptive name that will be familiar to experts yet easy to
google for the curious.

to repeat: i'm going to do some leg work so that the double and float prims
are portably exposed by ghc-prims (i've spoken with several ghc devs about
that, and they agree to its value, and thats a decision outside of scope of
the libraries purview), and I do hope we can to a consensus about putting
it in Num so that expert library authors can upgrade the guarantees that
they can provide end users without imposing any breaking changes to end
users.

A number of folks have brought up "but Num is broken" as a counter argument
to adding FMA support to Num. I emphatically agree  num is borken :), BUT!
I do also believe that fixing up Num prelude has the burden of providing a
whole cloth design for an alternative design that we can get broad
consensus/adoption with.  That will happen by dint of actually
experimentation and usage.

Point being, adding FMA doesn't further entrench current Num any more than
it already is, it just provides expert library authors with a transparent
way of improving the experience of their users with a free upgrade in
answer accuracy if used carefully. Additionally, when Num's "semiring ish
equational laws" are  framed with respect to approximate forwards/backwards
stability, there is a perfectly reasonable law for FMA. I am happy to spend
some time trying to write that up more precisely IFF that will tilt those
in opposition to being in favor.

I dont need FMA to be exposed by *prelude/base*, merely by *GHC.Num* as a
method therein for Num. If that constitutes a different and *more palatable
proposal*  than what people have articulated so far (by discouraging casual
use by dint of hiding) then I am happy to kick off a new thread with that
concrete design choice.

If theres a counter argument thats a bit more substantive than "Num is for
exact arithmetic" or "Num is wrong" that will sway me to the other side,
i'm all ears, but i'm skeptical of that.

I emphatically support those who are displeased with Num to prototype some
alternative designs in userland, I do think it'd be great to figure out a
new Num prelude we can migrate Haskell / GHC to over the next 2-5 years,
but again any such proposal really needs to be realized whole cloth before
it makes its way to being a libraries list proposal.

again, pardon the wall of text, i just really want to have nice things :)
-Carter

On Mon, May 4, 2015 at 2:22 PM, Levent Erkok <erkokl at gmail.com> wrote:

> I think `mulAdd a b c` should be implemented as `a*b+c` even for
> Double/Float. It should only be an "optmization" (as in modular
> arithmetic), not a semantic changing operation. Thus justifying the
> optimization.
>
> "fma" should be the "more-precise" version available for Float/Double. I
> don't think it makes sense to have "fma" for other types. That's why I'm
> advocating "mulAdd" to be part of "Num" for optimization purposes; and
> "fma" reserved for true IEEE754 types and semantics.
>
> I understand that Edward doesn't like this as this requires a different
> class; but really, that's the price to pay if we claim Haskell has proper
> support for IEEE754 semantics. (Which I think it should.) The operation is
> just different. It also should account for the rounding-modes properly.
>
> I think we can pull this off just fine; and Haskell can really lead the
> pack here. The situation with floats is even worse in other languages. This
> is our chance to make a proper implementation, and we have the right tools
> to do so.
>
> -Levent.
>
> On Mon, May 4, 2015 at 10:58 AM, Artyom <yom at artyom.me> wrote:
>
>>  On 05/04/2015 08:49 PM, Levent Erkok wrote:
>>
>> Artyom: That's precisely the point. The true IEEE754 variants where
>> precision does matter should be part of a different class. What Edward and
>> Yitz want is an "optimized" multiply-add where the semantics is the same
>> but one that goes faster.
>>
>> No, it looks to me that Edward wants to have a more precise operation in
>> Num:
>>
>> I'd have to make a second copy of the function to even try to see the
>> precision win.
>>
>> Unless I'm wrong, you can't have the following things simultaneously:
>>
>>    1. the compiler is free to substitute *a+b*c* with *mulAdd a b c*
>>    2. *mulAdd a b c* is implemented as *fma* for Doubles (and is more
>>    precise)
>>    3. Num operations for Double (addition and multiplication) always
>>    conform to IEEE754
>>
>>  The true IEEE754 variants where precision does matter should be part of
>> a different class.
>>
>> So, does it mean that you're fine with not having point #3 because people
>> who need it would be able to use a separate class for IEEE754 floats?
>>
>>
>
> _______________________________________________
> Libraries mailing list
> Libraries at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/libraries/attachments/20150504/1345f053/attachment-0001.html>