FFI: c/c++ struct on stack as an argument or return value

Tue Mar 18 01:37:27 UTC 2014

Yuras Shumovich <shumovichy at gmail.com>:
> I think the compiler is the right place. It is impossible to have
> efficient implementation in a library.
> 
> For dynamic wrapper (foreign import "wrapper" stuff) ghc generates piece
> of executable code at runtime. There are native implementations for a
> number of platforms, and libffi is used as a fall back for other
> platforms (see rts/Adjustor.c). AFAIK it is done that way because libffi
> is slower then native implementation.
> 
> Library implementation can't generate native dynamic wrapper, it has to
> use slow libffi.

When we first implemented the FFI, there was no libffi. Maintaining the adjustor code for all platforms is a PITA; hence, using libffi was a welcome way to improve portability.

Making the adjustor code more complicated by adding more functionality doesn’t sound like a good plan to me.

Besides, there are other overheads in addition to the actual marshalling in FFI calls and most of the time we are calling out to library functions for which the FFI call overhead is only a small portion of the runtime.

>> i think the crux of Manuel's point is mainly that any good proposal
>> has to
>> at least give a roadmap to support on all the various platforms etc
>> etc
> 
> I don't think you are expecting detailed schedule from me. Passing
> structure by value is possible on all platforms ghc supports, and it can
> be implemented for any particular platform if somebody is interested.
> 
> From my point of view, at this point it is more important to agree on
> the next question: do we want such functionality in ghc at all? I don't
> want to waste time on it if nobody wants to see it merged.

I still don’t see the benefit in further complicating an already murky corner of the compiler. Moreover, for this to make sense, it would need to work on all supported platforms. Unless you are volunteering to implement it on multiple platforms, this would mean, we’d use libffi for most platforms anyway. This brings me to my original point, a library or tool is the better place for this.

Manuel

PS: I’d happily accept language-c-inline patches for marshalling structs.

> On Sat, 2014-03-15 at 00:37 -0400, Carter Schonwald wrote:
>> I'm not opposing that, in fact, theres a GHC ticket discussing some stuff
>> related to this (related to complex numbers).
>> 
>> i think the crux of Manuel's point is mainly that any good proposal has to
>> at least give a roadmap to support on all the various platforms etc etc
>> 
>> 
>> On Sat, Mar 15, 2014 at 12:33 AM, Edward Kmett <ekmett at gmail.com> wrote:
>> 
>>> I don't care enough to fight and try to win the battle, but I just want to
>>> point out that Storable structs are far more brittle and platform dependent
>>> than borrowing the already correct platform logic for struct passing from
>>> libffi.
>>> 
>>> I do think the existing FFI extension made the right call under the 32 bit
>>> ABIs that were in use at the time it was defined. That said, with 64-bit
>>> ABIs saying that 2 32-bit ints should be passed in a single 64 bit
>>> register, you wind up with large chunks of third party APIs we just can't
>>> call out to directly any more, requiring many one-off manual C shims.
>>> 
>>> -Edward
>>> 
>>> 
>>> 
>>> 
>>> On Sat, Mar 15, 2014 at 12:17 AM, Carter Schonwald <
>>> carter.schonwald at gmail.com> wrote:
>>> 
>>>> indeed, its very very easy to do storable instances that correspond to
>>>> the struct type you want,
>>>> 
>>>> the ``with`` function in
>>>> http://hackage.haskell.org/package/base-4.6.0.1/docs/Foreign-Marshal-Utils.htmlforeigh.marshal.utils actually gets you most of the way there!
>>>> 
>>>> 
>>>> 
>>>> 
>>>> On Sat, Mar 15, 2014 at 12:00 AM, Manuel M T Chakravarty <
>>>> chak at cse.unsw.edu.au> wrote:
>>>> 
>>>>> Yuras,
>>>>> 
>>>>> I’m not convinced that the compiler is the right place for this kind of
>>>>> functionality. In fact, when we designed the Haskell FFI, we explicit
>>>>> decided against what you propose. There are a few reasons for this.
>>>>> 
>>>>> Firstly, compilers are complex beasts, and secondly, it takes a long
>>>>> time until a change in the compiler goes into production. Hence, as a
>>>>> general rule, it is advisable to move complexity from the compiler into
>>>>> libraries as this reduces compiler complexity. Libraries are less complex
>>>>> and changes can be rolled out much more quickly (it’s essentially a Hackage
>>>>> upload versus waiting for the next GHC and Haskell Platform release).
>>>>> 
>>>>> Thirdly, we have got the Haskell standard for a reason and modifying the
>>>>> compiler implies a language extension.
>>>>> 
>>>>> The design goal for the Haskell FFI was to provide the absolute minimum
>>>>> as part of the language and compiler, and to layer additional conveniences
>>>>> on top of that in the form of libraries and tools.
>>>>> 
>>>>> Have you considered the library or tool route?
>>>>> 
>>>>> Manuel
>>>>> 
>>>>> Yuras Shumovich <shumovichy at gmail.com>:
>>>>>> Hi,
>>>>>> 
>>>>>> Right now ghc's FFI doesn't support c/c++ structures.
>>>>>> 
>>>>>> Whenever we have foreign function that accepts or returns struct by
>>>>>> value, we have to create wrapper that accepts or returns pointer to
>>>>>> struct. It is inconvenient, but actually not a big deal.
>>>>>> 
>>>>>> But there is no easy workaround when you want to export haskell
>>>>> function
>>>>>> to use it with c/c++ API that requires structures to be passed by value
>>>>>> (Usually it is a callback in c/c++ API. You can't change it's
>>>>> signature,
>>>>>> and if it doesn't provide some kind of "void* userdata", then you are
>>>>>> stuck.)
>>>>>> 
>>>>>> I'm interested in fixing that. I'm going to start with 'foreign import
>>>>>> "wrapper" ...' stuff.
>>>>>> 
>>>>>> Calling conventions for passing c/c++ structures by value are pretty
>>>>>> tricky and platform/compiler specific. So initially I'll use libffi for
>>>>>> that (it will work when USE_LIBFFI_FOR_ADJUSTORS is defined, see
>>>>>> rts/Adjustor.c). It will allow me to explore design space without
>>>>>> bothering about low level implementation details. Later it could be
>>>>>> implemented for native (non-libffi) adjustors.
>>>>>> 
>>>>>> Is anybody interested it that? I appreciate any comments/ideas.
>>>>>> 
>>>>>> Right now I don't have clear design. It would be nice to support plain
>>>>>> haskell data types that are 1) not recursive, 2) has one constructor
>>>>> and
>>>>>> 3) contains only c/c++ types. But it doesn't work with c/c++ unions.
>>>>> Any
>>>>>> ideas are welcome.
>>>>>> 
>>>>>> An example how to use libffi with structures:
>>>>>> http://www.atmark-techno.com/~yashi/libffi.html#Structures
>>>>>> 
>>>>>> Thanks,
>>>>>> Yuras
>>>>>> 
>>>>>> 
>>>>>> _______________________________________________
>>>>>> ghc-devs mailing list
>>>>>> ghc-devs at haskell.org
>>>>>> http://www.haskell.org/mailman/listinfo/ghc-devs
>>>>> 
>>>>> _______________________________________________
>>>>> ghc-devs mailing list
>>>>> ghc-devs at haskell.org
>>>>> http://www.haskell.org/mailman/listinfo/ghc-devs
>>>>> 
>>>> 
>>>> 
>>>> _______________________________________________
>>>> ghc-devs mailing list
>>>> ghc-devs at haskell.org
>>>> http://www.haskell.org/mailman/listinfo/ghc-devs
>>>> 
>>>> 
>>> 
>> _______________________________________________
>> ghc-devs mailing list
>> ghc-devs at haskell.org
>> http://www.haskell.org/mailman/listinfo/ghc-devs
> 
> 
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://www.haskell.org/mailman/listinfo/ghc-devs