Making (useful subsets of) bytecode portable between targets

[Ben Gamari]

CCing Luite who has done this sort of thing in the context of GHCJS.


Shea Levy <shea at shealevy.com> writes:

> Hi all,
>
> I'm interested in implementing a general solution for TH during
> cross-compilation, which if my naive lack-of-understanding is correct
> will broadly involve the following three tasks:
>
> 1. Make the generation of byte code, or at least the subset needed for
>    useful TH use, target-independent

The bytecode machine itself (defined by ByteCodeInstr) is reasonably
target-independent. However, I suspect primops will be
problematic since they can appear in BCOs and are quite target dependent
(namely due to word size). I honestly don't know how much of a problem
this will be in practice; I tried loading a few example programs into
ghci -ddump-bcos and didn't see any PUSH_PRIMOP instructions, so perhaps
this won't be as problematic as I expect; it is worrisome though.

In general it's hard to identify "the subset needed for useful TH use"
but the fact that I can load the gitrev package (a fairly non-trivial
use of TH) into GHCi with no apparently platform dependent BCOs being
produced is promising.

> 2. Teach the external-interpreter to launch a build-native iserv when
>    cross-compiling

For this you'll want to have a careful look through the Binary instances
for the things that are sent over the wire to iserv. You'll find these
occur in libraries/ghci/*.hs.

The first thing to look for is places where the system word-size may
leak into the serialized representation (e.g. instances where Ints and
Words are being used). Many of these can probably just be narrowed into
32-bit representations (e.g. in ResolvedBCO, where I think it's fair to
assume that four-billion bytecode instructions is an unreasonable BCO
size).

Don't forget to consider the platform dependence of the "primitive"
encodings like the Binary instance for ByteString [1]; you'll need to
take care to replace these with platform-independent encodings.

[1] https://hackage.haskell.org/package/binary-0.8.4.1/docs/src/Data.Binary.Class.html#line-611

> 3. Teach cabal to compile dependencies and modules for the build and
>    target when cross-compiling and TH is used

You'll need to speak with the Cabal folks about this. However, it's
probably safe to ignore this for now; there's a lot of work to done
before you'll be in a position to start thinking about this.


> Of course, due to the generality of TH there will be code you can write
> that would be different in this approach from what you would get with a
> fully native compilation (e.g. due to GHC conditional compilation in the
> TH functions or FFI that does runtime host introspection), but since
> requiring a target device at build time is in many cases impractical (do
> you want to hook up an iPhone to your build farm?) and in some cases
> probably impossible (for targets without the resources to run a full GHC
> even if they can run GHC-compiled code) I think this is a reasonable
> restriction to require.
>
> My questions to the list are:
>
> * Is 1 above a pipe dream, keeping in mind that we are assuming the
>   availability of build-native versions of all dependencies and
>   already-compiled modules?

This sounds quite feasible to me.

> * Any pointers for getting started with 1?

See above. The first thing to do is to rid the Binary instances of
platform dependent encodings. This will be a bit of a game of
whack-a-mole, but it shouldn't be that hard.

Don't hesitate to ask if you get stuck!

Cheers,

- Ben

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 454 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20161121/65a4c097/attachment.sig>