Making (useful subsets of) bytecode portable between targets

[Simon Marlow]

On 21 November 2016 at 12:50, Shea Levy <shea at shealevy.com> wrote:

> Hi all,
>
> I'm interested in implementing a general solution for TH during
> cross-compilation, which if my naive lack-of-understanding is correct
> will broadly involve the following three tasks:
>
> 1. Make the generation of byte code, or at least the subset needed for
>    useful TH use, target-independent
> 2. Teach the external-interpreter to launch a build-native iserv when
>    cross-compiling
> 3. Teach cabal to compile dependencies and modules for the build and
>    target when cross-compiling and TH is used
>

The external interpreter was aimed at being able to run target code at
compile time.  I think what you have in mind is running *host* code at
compile-time instead.  There are a number of tricky things here.

- You have to build everything for the host as well as the target.  All the
libraries that GHC builds will need to be built in two flavours, and the
external build tools Cabal/Stack (as you mentioned) need to know about this.

- GHC itself will need to be able to generate code for multiple platforms.
We've done some work in this direction, but I don't think it's complete.
Maybe it's not all that far off, but there are still some platform #ifdefs
in the code.

- As you point out, the code we compile for the host platform will not
necessarily be the same as the target code, due to #ifdefs and other
platform differences.

- There is a point where the two platforms meet, when we're compiling a
module for the target platform that uses TH.  Every assumption that we made
about the target platform (from reading .hi files) must also be true of the
host platform, because we're about to run that code.  So the .hi files for
the host and target must be exactly the same.  One way this could easily go
wrong is if you use some API that's only available on the target platform
in some TH code.  GHC won't know that anything is wrong, but we'll get a
link error (or worse, a crash) when trying to run the TH code.

What this comes down to is that GHC doesn't have a clean separation between
compile-time execution and runtime execution, it assumes that they're the
same.  For a (slightly opinionated) perspective on this, see
http://blog.ezyang.com/2016/07/what-template-haskell-gets-wrong-and-racket-gets-right/
.

So what I'm saying is: you could probably make this work, for some slightly
flaky and brittle value of "work".  Doing it properly is also pretty hard
in the context of GHC, though, because it's a fairly fundamental upheaval.

My question would be: are you *sure* you can't run target code at compile
time?  Not even with an iphone simulator?

Cheers
Simon


Of course, due to the generality of TH there will be code you can write
> that would be different in this approach from what you would get with a
> fully native compilation (e.g. due to GHC conditional compilation in the
> TH functions or FFI that does runtime host introspection), but since
> requiring a target device at build time is in many cases impractical (do
> you want to hook up an iPhone to your build farm?) and in some cases
> probably impossible (for targets without the resources to run a full GHC
> even if they can run GHC-compiled code) I think this is a reasonable
> restriction to require.
>
> My questions to the list are:
>
> * Is 1 above a pipe dream, keeping in mind that we are assuming the
>   availability of build-native versions of all dependencies and
>   already-compiled modules?
> * Any pointers for getting started with 1?
> * Anything I'm missing?
>
> Thanks,
> Shea
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20161123/330bd173/attachment.html>