Proposal: Add "fma" to the RealFloat class

Tue May 5 14:23:26 UTC 2015

On Tue, May 5, 2015 at 8:16 AM, Carter Schonwald <carter.schonwald at gmail.com
> wrote:

> To clarify: I think theres a bit of an open design question how the
> explicitly moded api would look. I'd suspect it'll look somewhat like Ed's
> AD lib, and should be in a  userland library I think.
>

Another concern here is laziness.  What happens when you force a thunk of
type Double inside a "withRoundingMode" kind of construct?

-Jan

>  On May 5, 2015 7:40 AM, "Carter Schonwald" <carter.schonwald at gmail.com>
> wrote:
>
>> Hey Levent,
>> I actually looked into how to do rounding mode setting a while ago, and
>> the conclusion I came to is that those can simply be ffi calls at the top
>> level that do a sort of with mode bracketing.  Or at least I'm not sure if
>> setting the mode in an inner loop is a good idea.
>>
>> That said, you are making a valid point, and I will investigate to what
>> extent compiler support is useful for the latter. If bracketed mode setting
>> and unsetting has a small enough performance overhead, adding support in
>> ghc primops would be worth while.  Note that those primops would have to be
>> modeled as doing something thats like io or st, so that when mode switches
>> happen can be predictable.  Otherwise CSE and related optimizations could
>> result in evaluating the same code in the wrong mode.  I'll think through
>> how that can be avoided, as I do have some ideas.
>>
>> I suspect mode switching code will wind up using new type wrapped floats
>> and doubles that have a phantom index for the mode, and something like
>> "runWithModeFoo:: Num a => Mode m->(forall s . Moded s a )  -> a"    to
>> make sure mode choices happen predictably.  That said, there might be a
>> better approach that we'll come to after some experimenting
>> On May 5, 2015 12:54 AM, "Levent Erkok" <erkokl at gmail.com> wrote:
>>
>>> Carter: Wall of text is just fine!
>>>
>>> I'm personally happy to see the results of your experiment. In
>>> particular, the better "code-generation" facilities you add around
>>> floats/doubles that map to the underlying hardware's native instructions,
>>> the better. When we do have proper IEEE floats, we shall surely need all
>>> that functionality.
>>>
>>> While you're working on this, if you can also watch out for how rounding
>>> modes can be integrated into the operations, that would be useful as well.
>>> I can see at least two designs:
>>>
>>>    * One where the rounding mode goes with the operation: `fpAdd
>>> RoundNearestTiesToEven 2.5 6.4`. This is the "cleanest" and the functional
>>> solution, but could get quite verbose; and might be costly if the
>>> implementation changes the rounding-mode at every issue.
>>>
>>>    * The other is where the operations simply assume the
>>> RoundNearestTiesToEven, but we have lifted IO versions that can be modified
>>> with a "with" like construct:  `withRoundingMode RoundTowardsPositive $
>>> fpAddRM 2.5 6.4`. Note that `fpAddRM` (*not* `fpAdd` as before) will have
>>> to return some sort of a monadic value (probably in the IO monad) since
>>> it'll need to access the rounding mode currently active.
>>>
>>> Neither choice jumps out at me as the best one; and a hybrid might also
>>> be possible. I'd love to hear any insight you gain regarding rounding-modes
>>> during your experiment.
>>>
>>> -Levent.
>>>
>>> On Mon, May 4, 2015 at 7:54 PM, Carter Schonwald <
>>> carter.schonwald at gmail.com> wrote:
>>>
>>>> pardon the wall of text everyone, but I really want some FMA tooling :)
>>>>
>>>> I am going to spend some time later this week and next adding FMA
>>>> primops to GHC and playing around with different ways to add it to Num
>>>> (which seems pretty straightforward, though I think we'd all agree it
>>>> shouldn't be exported by Prelude). And then depending on how Yitzchak's
>>>> reproposal  of that exactly goes (or some iteration thereof) we can get
>>>> something useful/usable into 7.12
>>>>
>>>> i have codes (ie *dotproducts*!!!!!) where a faster direct FMA for *exact
>>>> numbers*, and a higher precision FMA for *approximate numbers *(*ie
>>>> floating point*),  and where I cant sanely use FMA if it lives
>>>> anywhere but Num unless I rub typeable everywhere and do runtime type
>>>> checks for applicable floating point types, which kinda destroys
>>>> parametrically in engineering nice things.
>>>>
>>>> @levent: ghc doesn't do any optimization for floating point arithmetic
>>>> (aside from 1-2 very simple things that are possibly questionable), and
>>>> until ghc has support for precisly emulating high precision floating point
>>>> computation in a portable way, probably wont have any interesting floating
>>>> point computation.  Mandating that fma a b c === a*b+c for inexact number
>>>> datatypes doesn't quite make sense to me. Relatedly, its a GOOD thing ghc
>>>> is conservative about optimizing floating point, because it makes doing
>>>> correct stability analyses tractable!  I look forward to the day that GHC
>>>> gets a bit more sophisticated about optimizing floating point computation,
>>>> but that day is still a ways off.
>>>>
>>>> relatedly: FMA for float and double are not generally going to be
>>>> faster than the individual primitive operations, merely more accurate when
>>>> used carefully.
>>>>
>>>> point being*, i'm +1 on adding some manner of FMA operations to Num*
>>>> (only sane place to put it where i can actually use it for a general use
>>>> library) and i dont really care if we name it fusedMultiplyAdd,
>>>> multiplyAndAdd accursedFusionOfSemiRingOperations, or fma. i'd favor
>>>> "fusedMultiplyAdd" if we want a descriptive name that will be familiar to
>>>> experts yet easy to google for the curious.
>>>>
>>>> to repeat: i'm going to do some leg work so that the double and float
>>>> prims are portably exposed by ghc-prims (i've spoken with several ghc devs
>>>> about that, and they agree to its value, and thats a decision outside of
>>>> scope of the libraries purview), and I do hope we can to a consensus about
>>>> putting it in Num so that expert library authors can upgrade the guarantees
>>>> that they can provide end users without imposing any breaking changes to
>>>> end users.
>>>>
>>>> A number of folks have brought up "but Num is broken" as a counter
>>>> argument to adding FMA support to Num. I emphatically agree  num is borken
>>>> :), BUT! I do also believe that fixing up Num prelude has the burden of
>>>> providing a whole cloth design for an alternative design that we can get
>>>> broad consensus/adoption with.  That will happen by dint of actually
>>>> experimentation and usage.
>>>>
>>>> Point being, adding FMA doesn't further entrench current Num any more
>>>> than it already is, it just provides expert library authors with a
>>>> transparent way of improving the experience of their users with a free
>>>> upgrade in answer accuracy if used carefully. Additionally, when Num's
>>>> "semiring ish equational laws" are  framed with respect to approximate
>>>> forwards/backwards stability, there is a perfectly reasonable law for FMA.
>>>> I am happy to spend some time trying to write that up more precisely IFF
>>>> that will tilt those in opposition to being in favor.
>>>>
>>>> I dont need FMA to be exposed by *prelude/base*, merely by *GHC.Num*
>>>> as a method therein for Num. If that constitutes a different and *more
>>>> palatable proposal*  than what people have articulated so far (by
>>>> discouraging casual use by dint of hiding) then I am happy to kick off a
>>>> new thread with that concrete design choice.
>>>>
>>>> If theres a counter argument thats a bit more substantive than "Num is
>>>> for exact arithmetic" or "Num is wrong" that will sway me to the other
>>>> side, i'm all ears, but i'm skeptical of that.
>>>>
>>>> I emphatically support those who are displeased with Num to prototype
>>>> some alternative designs in userland, I do think it'd be great to figure
>>>> out a new Num prelude we can migrate Haskell / GHC to over the next 2-5
>>>> years, but again any such proposal really needs to be realized whole cloth
>>>> before it makes its way to being a libraries list proposal.
>>>>
>>>>
>>>> again, pardon the wall of text, i just really want to have nice things
>>>> :)
>>>> -Carter
>>>>
>>>>
>>>> On Mon, May 4, 2015 at 2:22 PM, Levent Erkok <erkokl at gmail.com> wrote:
>>>>
>>>>> I think `mulAdd a b c` should be implemented as `a*b+c` even for
>>>>> Double/Float. It should only be an "optmization" (as in modular
>>>>> arithmetic), not a semantic changing operation. Thus justifying the
>>>>> optimization.
>>>>>
>>>>> "fma" should be the "more-precise" version available for Float/Double.
>>>>> I don't think it makes sense to have "fma" for other types. That's why I'm
>>>>> advocating "mulAdd" to be part of "Num" for optimization purposes; and
>>>>> "fma" reserved for true IEEE754 types and semantics.
>>>>>
>>>>> I understand that Edward doesn't like this as this requires a
>>>>> different class; but really, that's the price to pay if we claim Haskell
>>>>> has proper support for IEEE754 semantics. (Which I think it should.) The
>>>>> operation is just different. It also should account for the rounding-modes
>>>>> properly.
>>>>>
>>>>> I think we can pull this off just fine; and Haskell can really lead
>>>>> the pack here. The situation with floats is even worse in other languages.
>>>>> This is our chance to make a proper implementation, and we have the right
>>>>> tools to do so.
>>>>>
>>>>> -Levent.
>>>>>
>>>>> On Mon, May 4, 2015 at 10:58 AM, Artyom <yom at artyom.me> wrote:
>>>>>
>>>>>>  On 05/04/2015 08:49 PM, Levent Erkok wrote:
>>>>>>
>>>>>> Artyom: That's precisely the point. The true IEEE754 variants where
>>>>>> precision does matter should be part of a different class. What Edward and
>>>>>> Yitz want is an "optimized" multiply-add where the semantics is the same
>>>>>> but one that goes faster.
>>>>>>
>>>>>> No, it looks to me that Edward wants to have a more precise operation
>>>>>> in Num:
>>>>>>
>>>>>> I'd have to make a second copy of the function to even try to see the
>>>>>> precision win.
>>>>>>
>>>>>> Unless I'm wrong, you can't have the following things simultaneously:
>>>>>>
>>>>>>    1. the compiler is free to substitute *a+b*c* with *mulAdd a b c*
>>>>>>    2. *mulAdd a b c* is implemented as *fma* for Doubles (and is
>>>>>>    more precise)
>>>>>>    3. Num operations for Double (addition and multiplication) always
>>>>>>    conform to IEEE754
>>>>>>
>>>>>>  The true IEEE754 variants where precision does matter should be
>>>>>> part of a different class.
>>>>>>
>>>>>> So, does it mean that you're fine with not having point #3 because
>>>>>> people who need it would be able to use a separate class for IEEE754 floats?
>>>>>>
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Libraries mailing list
>>>>> Libraries at haskell.org
>>>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
>>>>>
>>>>>
>>>>
>>>
> _______________________________________________
> Libraries mailing list
> Libraries at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/libraries/attachments/20150505/84c4668f/attachment-0001.html>