FFI - some comments

5 Apr 2001 19:58:34 GMT

Thu, 5 Apr 2001 07:45:44 -0700 (PDT), Ronald Legere <rjljr2@yahoo.com> pisze:

> KDirect: Looked at it abit. Pretty sparsely documented. I could not
> figure out how it fits in with the latest tools. How does QForeign
> relate to Foreign? What is an hsc file? Also it is pretty limitted
> in the marshalling.  No callbacks etc.

Here is the story of QForeign, hsc and KDirect.

One year ago the core 'foreign' declarations were already stable and
implemented in ghc (for sure) and nhc (probably already at that time).
The FFI documentation said that support for more complex marshalling
is delegated to separate tools.

Available tools, namely HDirect and Manuel's C2HS, were too heavy for
my taste. But Sven's modules provided by ghc: FFI (later renamed to
Foreign) and Marshal, were insufficient.

I realized that a few more functions would make the basic FFI
usable directly, without sophisticated preprocessors. I also wanted
to experiment with typed 'Ptr a' instead of amorphic 'Addr'. So I
merged ideas from ghc's module FFI and C2HS with my own FFI vision,
and QForeign was born.

QForeign evolved. The basic style of calling a C function while
marshalling arguments and results changed a few times. It got examples.
It suggested how to handle structs and enums. It became my Unicode
laboratory.

Since writing lots of small C functions for getting struct fields and
constants was painful, I wrote glue-hsc: a simple preprocessor which
translates a Haskell module with embedded C bits into a C program which
outputs a real Haskell module. It allows embedding snippets of C code
which are extracted into .c and .h files to be compiled separately.
The source file extension .hsc means: .hs + bits of C included.

QForeign was following changes in ghc and it also influenced ghc.
ghc got typed pointers, alloca, withForeignObj, C array support, hsc2hs
(renamed from glue-hsc), errno handling, C string support interface.
As soon as some QForeign's feature got into ghc, QForeign switched to
use it from ghc instead of providing it itself. If a feature mutated
during its path to ghc, QForeign reflected the change. ghc's FFI
support and my FFI vision were slowly converging.

I had to write a networking program at the University, so of course
I decided to use Haskell. That's why QForeign has pcap in examples.
I asked my admin to install ghc in students' lab. The latest stable
version was 4.08.1, so I #ifdefed parts of QForeign to support
ghc-4.08* too by providing its own typed pointers and Storable again
(no Unicode, no --add-package).

Supporting several ghc versions at once happened to be easy (hsc2hs
delegates #ifdefs to cpp). So I tried to port QForeign to nhc too
and it worked, after some bugs that I reported were fixed. So I
also thought about supporting hbc, which would require writing a
preprocessor which translates 'foreign' declarations to appropriate
hbc's magic, but hbc is dead, so I forgot it. Since Hugs recently got
'foreign' declarations, it might be possible to port QForeign to Hugs
in future, but its foreign module management looks hard.

QForeign and FFI modules provided by ghc-5.00 (to be released really
soon) are now very close, after Manuel put some marshalling utilities
into ghc and I backported them to QForeign as usual. The largest
difference is that QForeign makes real use of Unicode (when the
compiler has wide Chars) by converting most strings exchanged with
the world between the specified encoding (default: locale dependent)
and Haskell's Unicode, even though Unicode support modules are still
experimental and transparent conversion on Handle IO is temporarily
hacked on top of original Handle functions. In essence QForeign
provides the interface of ghc-5.00's FFI on ghc >= 4.08 and nhc98.

My friend Wojciech Moczydlowski <khaliff@astercity.net> asked me what
is the currently recommended way of interfacing between Haskell and
C. I showed him how do I do it.

He said that Greencard and HDirect were easier: you didn't have to
write marshalling code around each function, but all arguments were
magically coverted basing on a single type declaration of the function.

I replied that my solution is more general, especially if the Haskell's
interface of a C module doesn't want to exactly mirror the original C
interface, and that it's impossible to match everything automatically,
and that it's not that painful to write function wrappers by hand.

He said that a tool could handle simple cases automatically, leaving
only hard types to the programmer.

I said that I don't know how express a C interface to archieve
this, and that C libraries are too complex to be wrapped in Haskell
automatically, and thus I can't provide anything more sophisticated
than hsc2hs.

So he created KDirect.

KDirect currently works on top of QForeign and hsc2hs. I assume that
in future it will be able to use FFI modules provided by ghc >= 5.00,
and that eventually all Haskell implementations will provide common
FFI modules.

-- 
 __("<  Marcin Kowalczyk * qrczak@knm.org.pl http://qrczak.ids.net.pl/
 \__/
  ^^                      SYGNATURA ZASTĘPCZA
QRCZAK