FFI proposal: allow some control over the scope of C header files

[Duncan Coutts]

On Sun, 2006-04-23 at 17:26 -0400, Manuel M T Chakravarty wrote:
> Duncan Coutts:
> > On Fri, 2006-04-21 at 09:32 -0400, Manuel M T Chakravarty wrote:
> > 
> > > > I think we'd want to be able to specify that a C header file not
> > > > "escape" a module boundary and probably we'd also want to be able to ask
> > > > that it not escape a package boundary (though this may be beyond the H'
> > > > spec since Haskell does not talk about packages).
> > > 
> > > The H98 standard already specifies a NOINLINE pragma for any function:
> > > 
> > >   http://haskell.org/onlinereport/pragmas.html
> > > 
> > > The simplest solution is to ensure that all Haskell compilers implement
> > > this pragma properly for foreign imported functions.  If you want finer
> > > control over where inlining takes place, then maybe the pragma should be
> > > extended to provide that finer control.
> > 
> > I don't think we need to generalise the problem to all function
> > inlinings. There are specific practical problems caused by inlining
> > foreign calls that are not a problem for ordinary Haskell functions.
> 
> Inlining of foreign functions causes extra problems, but generally
> inlining is a concern; so, if we can use the same mechanisms, we get a
> simpler language.

True, though with a special case mechanism we can make automatic checks
possible/easier.

> > > Besides, the standard so far doesn't cover command line options at all.
> > > So, there is the more general question of whether it should.
> > 
> > I don't think we need to specify the command line interface. The
> > required headers can be put in the module.
> 
> That's ok with me.  I was just pointing out that many of the problems
> and/or lack of understanding of users that we are seeing has to do with
> the use of command line options.  We simply cannot address this unless
> the standard covers command line options.

Under my hypothetical scheme the ghc command line method would be
equivalent to putting it in the module and could be checked the same
way.

> > What I really want is for the issue of header scope to be something that
> > can be checked by the compiler. As a distro packager I see far too many
> > people getting it wrong because they don't understand the issue. If we
> > could declare the intended scope of the header files then 1. people
> > would think about and 2. if they got it wrong it'd be checkable because
> > the compiler would complain.
> 
> Whether or not the compiler can check for wrong use, seems to me
> independent of whether we use inline pragmas or any other syntax.  GHC
> could very well check some of these things today.  It just doesn't.  Do
> you propose to make such checks mandatory in the standard?

That'd be nice, though I can see that it is more work.

> We are having two issues here:
> 
> (1) Specification of which functions need what headers and whether 
>     these functions can be inlined.
> (2) Let the compiler spot wrong uses of header files.
> 
> These two issues are largely independent.

Yes, ok.

>   Re (1), I dislike new syntax
> (or generally any additions to the language) and prefer using existing
> mechanisms as far as possible.  The reason is simply that Haskell is
> already very complicated.  Haskell' will be even more complicated.
> Hence, we must avoid any unnecessary additions.

Sure.

> Re (2), I am happy to discuss what kind of checks are possible, but I am
> worried that it'll be hard to check for everything without assistance
> from cabal, which I don't think will be part of Haskell'.

I think it can be checked without cabal. Outline: suppose we use a
module level granularity (I know jhc proposes to use a finer
granularity) so we track which C header files are needed to compile
which modules.

A FFI decl specifying a header file makes that module need that header.
Then transitively each module that imports that module needs that header
too. We can only stop the header leaking out of the module/package by
specifying NOINLINE on the imported function (or using some additional
syntax s I originally suggested).

So now it's easy to check what headers are needed to to compile any
module. Then we probably need to rely on an external mechanism (eg cabal
or the user) to make sure that all these headers are available - but at
least we can check that the user has done it right.

So it's at this point that issue (1) & (2) become related. If we say
that a header file transitively infects every client module then it
effectively bans private header files and so we need some mechanism to
limit the scope of header files to allow them again (like NOINLINE).

Eg with c2hs, I think that in theory we should be installing every .h
file that c2hs generates for each module with the library package. I've
never seen anyone actually do that (Cabal's c2hs support doesn't do that
for example).

> Re the concern about wrong use: FFI programming is a minefield.  We will
> never be able to make it safe.  So, I am reluctant to complicate the
> language just to make it (maybe) a little safer.  What IMHO will be far
> more effective is a good tutorial on FFI programming.

While it's true that it's easy to shoot yourself in the foot with the
FFI, this is a particular issue that people get wrong very often -
mostly because it does work most of the time and so they never find the
problem by testing (unlike most other FFI bugs which will give you a
segfault fairly quickly)

Duncan