[GHC] #9197: FFI types should be usable in foreign import decls without revealing representations

[GHC]

#9197: FFI types should be usable in foreign import decls without revealing
representations
-------------------------------------+------------------------------------
        Reporter:  duncan            |            Owner:
            Type:  bug               |           Status:  new
        Priority:  normal            |        Milestone:
       Component:  Compiler (FFI)    |          Version:  7.8.2
      Resolution:                    |         Keywords:
Operating System:  Unknown/Multiple  |     Architecture:  Unknown/Multiple
 Type of failure:  None/Unknown      |       Difficulty:  Unknown
       Test Case:                    |       Blocked By:
        Blocking:                    |  Related Tickets:
-------------------------------------+------------------------------------
Description changed by duncan:

Old description:

> This is another go at #5529 but with, we hope, clearer and better
> arguments.
>
> The original FFI spec defined a whole bunch of FFI types that are
> acceptable in foreign import/export declarations, including lots of
> things defined in Foreign.C.*. The original spec defined these as
> abstract types.
>
> The original FFI spec also allowed for automatic newtype unwrapping in
> foreign declarations. In 2009 Simon PJ [pointed out
> http://www.haskell.org/pipermail/haskell-prime/2009-February/002726.html]
> that automatic newtype unwrapping breaks abstraction for user-defined
> types. In that same thread [I suggested http://www.haskell.org/pipermail
> /haskell-prime/2009-February/002727.html] that the breaking of
> abstraction for user types should be fixed, but that the intent of the
> FFI spec to have the FFI types as abstract should be kept. Manuel [argued
> http://www.haskell.org/pipermail/haskell-prime/2009-February/002729.html]
> that the C FFI types should actually not be abstract, and thus no special
> case for them was needed.
>
> On the basis of that thread the decision in #5529 was to only allow FFI
> decls to look through newtypes when the constructor is available, plus
> for a few types GHC uses to represent the C FFI types, like Int32 etc.
>
> Johan and I want to argue that it still makes sense as a user to not be
> forced into knowing the representation of the C FFI types to be able to
> use them. This is similar to but slightly weaker than the rationale of
> the FFI spec originally making those types abstract. Manuels argument is
> essentially that sometimes you need to know concrete types when those can
> change, e.g. CInt might be Int32 or Int64 and you should be free to
> depend on that difference.
>
> Manuel's argument is fine when the "concrete" representation exists as
> yet another portable type, as is the case for Int32. Of course we don't
> insist that the representation for Int or Int32 be exposed, in large part
> because they cannot have a Haskell implementation independent
> representation. The important point is
>
> Johan and I are defining/writing a portable low level ByteArray library
> that could be considered alongside the FFI's definition of some of the
> Foreign.* libraries: it defines and API for some low level types and
> functions but different Haskell implementations will implement them
> differently.
>
> For GHC we implement `data ByteArray = BA ByteArray#`. We want to make
> this an FFI type, in the same sense as the FFI spec defines the C types
> to be FFI types. (You could consider our library as a Haskell extension
> and implementations following this extension will support this type for
> FFI in a similar way to the FFI spec). Of course for this implementation
> of `ByteArray` the representation crosses the boundary into
> implementation dependent, unlike the case for `newtype CInt = CInt Int32`
> where both are still portable types.
>
> So, we think the right thing to do is to again be able to declare certain
> types (those defined in the FFI spec and its extensions) as being valid
> types to use in FFI decls, without their constructors being available.
> Thus users could choose to import `CInt` only (i.e. without its
> constructor) and use it. We are not arguing against Manuel's point that
> it's sometimes useful to not hide representations where that is possible
> (like for CInt being an Int32 or 64). So we are not asking that the C FFI
> types be made abstract again, just that it be possible to use them (and
> our new type) as FFI types without user access to the constrctor.
>
> For GHC, my suggestion for a reasonable way to do this is with a pragma.
> The pragma would say that this single-constrcutor single-field newtype or
> data is an FFI type, and that GHC may perform the usual newtype
> unwrapping even when the constructor is not in scope at the FFI decl
> site. We would then use this pragma on Int, Int32 and other core GHC (and
> FFI) types, and also on the various C FFI types. And finally, we would
> also do this on our ByteArray type.
>
> Note that it would have to cover data as well as newtype to be able to
> work for Int and other boxed wrappers of unboxed types.

New description:

 This is another go at #5529 but with, we hope, clearer and better
 arguments.

 The original FFI spec defined a whole bunch of FFI types that are
 acceptable in foreign import/export declarations, including lots of things
 defined in Foreign.C.*. The original spec defined these as abstract types.

 The original FFI spec also allowed for automatic newtype unwrapping in
 foreign declarations. In 2009 Simon PJ [http://www.haskell.org/pipermail
 /haskell-prime/2009-February/002726.html pointed out] that automatic
 newtype unwrapping breaks abstraction for user-defined types. In that same
 thread [http://www.haskell.org/pipermail/haskell-
 prime/2009-February/002727.html I suggested] that the breaking of
 abstraction for user types should be fixed, but that the intent of the FFI
 spec to have the FFI types as abstract should be kept. Manuel
 [http://www.haskell.org/pipermail/haskell-prime/2009-February/002729.html
 argued] that the C FFI types should actually not be abstract, and thus no
 special case for them was needed.

 On the basis of that thread the decision in #5529 was to only allow FFI
 decls to look through newtypes when the constructor is available, plus for
 a few types GHC uses to represent the C FFI types, like Int32 etc.

 Johan and I want to argue that it still makes sense as a user to not be
 forced into knowing the representation of the C FFI types to be able to
 use them. This is similar to but slightly weaker than the rationale of the
 FFI spec originally making those types abstract. Manuels argument is
 essentially that sometimes you need to know concrete types when those can
 change, e.g. CInt might be Int32 or Int64 and you should be free to depend
 on that difference.

 Manuel's argument is fine when the "concrete" representation exists as yet
 another portable type, as is the case for Int32. Of course we don't insist
 that the representation for Int or Int32 be exposed, in large part because
 they cannot have a Haskell implementation independent representation. The
 important point is

 Johan and I are defining/writing a portable low level ByteArray library
 that could be considered alongside the FFI's definition of some of the
 Foreign.* libraries: it defines and API for some low level types and
 functions but different Haskell implementations will implement them
 differently.

 For GHC we implement `data ByteArray = BA ByteArray#`. We want to make
 this an FFI type, in the same sense as the FFI spec defines the C types to
 be FFI types. (You could consider our library as a Haskell extension and
 implementations following this extension will support this type for FFI in
 a similar way to the FFI spec). Of course for this implementation of
 `ByteArray` the representation crosses the boundary into implementation
 dependent, unlike the case for `newtype CInt = CInt Int32` where both are
 still portable types.

 So, we think the right thing to do is to again be able to declare certain
 types (those defined in the FFI spec and its extensions) as being valid
 types to use in FFI decls, without their constructors being available.
 Thus users could choose to import `CInt` only (i.e. without its
 constructor) and use it. We are not arguing against Manuel's point that
 it's sometimes useful to not hide representations where that is possible
 (like for CInt being an Int32 or 64). So we are not asking that the C FFI
 types be made abstract again, just that it be possible to use them (and
 our new type) as FFI types without user access to the constrctor.

 For GHC, my suggestion for a reasonable way to do this is with a pragma.
 The pragma would say that this single-constrcutor single-field newtype or
 data is an FFI type, and that GHC may perform the usual newtype unwrapping
 even when the constructor is not in scope at the FFI decl site. We would
 then use this pragma on Int, Int32 and other core GHC (and FFI) types, and
 also on the various C FFI types. And finally, we would also do this on our
 ByteArray type.

 Note that it would have to cover data as well as newtype to be able to
 work for Int and other boxed wrappers of unboxed types.

--

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/9197#comment:1>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler