Making (useful subsets of) bytecode portable between targets

Fri Nov 25 11:11:25 UTC 2016

On 25 November 2016 at 07:23, Moritz Angermann <moritz at lichtzwerge.de>
wrote:

[snip]

> To get this back on topic, if we have a architecture independent
> interpretable bytecode,
> for ghc, could we sidestep the runner solution altogether and have TH for
> the target
> be evaluated on the host?  Is this what Shea initially wanted to go after?
>

Yes, but architecture-independent bytecode is the least of the problems.
Doing this properly is a really big change.

We basically have two worlds: first, the compile-time world. In this world,
we need all the packages and modules of the current package built for the
host platform. Secondly, we need the runtime world, with all the packages
and modules of the current package cross-compiled for the target platform.

When compiling a module that uses TH, we need to
- compile it as if we were compiling for the host platform, reading .hi
files from the host world
- run the TH code in the host world
- restart the compilation, using the .hi files from the target world, and
the results of running the splices

But even this isn't going to be enough.  What if your code imports some
modules that are only available on the runtime platform? iOS APIs, for
example.  The right thing is to have a clean separation between runtime
imports and compile-time imports.  Perhaps we just annotate some imports to
say they aren't needed at compile-time for running the TH code.  but then
we also need compile-time vs. runtime build-depends in our .cabal files,
and so on.

This is just off the top of my head, I'm sure there are more complexities I
haven't thought about.

Its a big project, but ultimately we do have to tackle it, because it's the
right thing to do.  Anyone interested in working on this?  Maybe start a
new proposal to flesh out the details.

Cheers
Simon

> cheers,
>  moritz
>
> > On Nov 25, 2016, at 2:38 PM, Manuel M T Chakravarty <
> chak at justtesting.org> wrote:
> >
> > Ok, I am not saying that it is technical impossible. I am saying that it
> is *impractical*.
> >
> > Imagine Travis CI needing to run stuff on my phone that is attached to
> my Mac (if we are lucky), which is behind NAT somewhere in Australia.
> >
> > Running stuff in the simulator during a build would be pretty awkward,
> but running it on the device is not practical.
> >
> > Manuel
> >
> > PS: BTW, shipping binary code to the device means it has to be code
> signed using a distribution profile of a registered developer. That is one
> thing if Xcode does all the magic behind the scenes, but do you really want
> to make that part of the GHC build process?
> >
> >> Edward Z. Yang <ezyang at MIT.EDU>:
> >>
> >> At least for Travis, you can generate a private key that only Travis
> >> has access to, and use this to authenticate access to the runner.
> >> See https://docs.travis-ci.com/user/encryption-keys/
> >>
> >> Edward
> >>
> >> Excerpts from Manuel M T Chakravarty's message of 2016-11-24 16:38:34
> +1100:
> >>> If you use Travis CI or such, do you really want to have a runner
> accessible from an arbitrary host on the Internet?
> >>>
> >>>> Moritz Angermann <moritz at lichtzwerge.de>:
> >>>>
> >>>> It's certainly far from ideal, but for CI, what obstacles are there
> besides needing a runner accessible from cross compiling machine?
> >>>>
> >>>> E.g. Start the runner app on an iPhone plugged in into a USB power
> source and leave it there?
> >>>>
> >>>> Sent from my iPhone
> >>>>
> >>>>> On 24 Nov 2016, at 12:42 PM, Manuel M T Chakravarty <
> chak at justtesting.org> wrote:
> >>>>>
> >>>>> Sorry, but I don’t think running on the device is practical. How do
> you want to do CI, for example?
> >>>>>
> >>>>> Manuel
> >>>>>
> >>>>>> Moritz Angermann <moritz at lichtzwerge.de>:
> >>>>>>
> >>>>>>
> >>>>>>> On Nov 23, 2016, at 7:50 PM, Simon Marlow <marlowsd at gmail.com>
> wrote:
> >>>>>>>
> >>>>>>> […]
> >>>>>>>
> >>>>>>> My question would be: are you *sure* you can't run target code at
> compile time?  Not even with an iphone simulator?
> >>>>>>
> >>>>>> This should be possible. However for proper development one would
> need to run on the
> >>>>>> device (iPhone, iPad, …) for armv7 or arm64, as the Simulator is
> i386 or x86_64.
> >>>>>>
> >>>>>> There is a bit of additional engineering required here to get the
> shipping of
> >>>>>> code from ghc to the runner on the target required (e.g. via
> network).  As executing
> >>>>>> and controlling applications on the actual hardware is limited, I
> guess a custom
> >>>>>> ghc-runner application would have to be manually started on the
> device, which could
> >>>>>> trivially be discovered using bonjour/zeroconf (or just giving ghc
> the host:port information).
> >>>>>>
> >>>>>> In general though, the runner does not have to obey all the
> restrictions apple puts
> >>>>>> onto app-store distributed apps, as I expect that everyone could
> build and install
> >>>>>> the runner themselves when intending to do iOS development with ghc.
> >>>>>>
> >>>>>> cheers,
> >>>>>> moritz
> >>>>>> _______________________________________________
> >>>>>> ghc-devs mailing list
> >>>>>> ghc-devs at haskell.org
> >>>>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
> >>>>>
> >>>>
> >>>
> >> _______________________________________________
> >> ghc-devs mailing list
> >> ghc-devs at haskell.org
> >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
> >
> > _______________________________________________
> > ghc-devs mailing list
> > ghc-devs at haskell.org
> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20161125/2c8da924/attachment.html>