GHCJS now runs Template Haskell on node.js - Any interest in out of process TH for general cross compilation?

[Luite Stegeman]

hi all,

I've added some code [1] [2] to GHCJS to make it run Template Haskell code
on node.js, rather than using the GHC linker. GHCJS has supported TH for a
long time now, but so far always relied on native (host) code for it. This
is the main reason that GHCJS always builds native and JavaScript code for
everything (another is that Cabal Setup.hs scripts need to be compiled to
some host-runnable form, but that can also be JavaScript if you have
node.js)

Now besides the compiler having to do twice the work, this has some other
disadvantages:

- Our JavaScript code has the same dependencies (packages) as native code,
which means packages like unix or Win32 show up somewhere, depending on the
host environment. This also limits our options in choosing JS-specific
packages.
- The Template Haskell code runs on the host environment, which might be
slightly different from the target, for example in integer size or
operating system specific constants.

Moreover, building native code made the GHCJS installation procedure more
tricky, making end users think about libgmp or libiconv locations, since it
basically required the same preparation as building GHC from source. This
change will make installing much easier and more reliable (we still have to
update the build scripts).

How it works is pretty simple:

- When any code needs to be run on the target (hscCompileCoreExpr, through
the Hooks API new in GHC 7.8), GHCJS starts a node.js process with the
thrunner.js [3] script,
- GHCJS sends its RTS and the Template Haskell server code [1] to node.js,
the script starts a Haskell thread running the server,
- for every splice, GHCJS compiles it to JavaScript and links it using its
incremental linking functionality. The code for the splice, including
dependencies that have not yet been sent to the runner (for earlier
splices), is then sent in a RunTH [4] message,
- the runner loads and runs the code in the Q monad, can send queries to
GHCJS for reification,
- the runner sends back the result as a serialized Template Haskell AST
(using GHC.Generics for the Binary instances).

All Template Haskell functionality is supported, including recent additions
for reifying modules and annotations. I still need to clean up and push the
patches for the directory and process packages, but after that, the TH code
can read/write files, run processes and interact with them and make network
connections, all through node.js.

Now since this approach is in no way specific to JavaScript, I was
wondering if there's any interest in getting this functionality into GHC
7.10 for general cross compilation. The runner would be a native (target)
program with dynamic libraries (or object files) being sent over to the
target machine (or emulator) for the splices.

Thanks to Andras Slemmer from Prezi who helped build the initial proof of
concept (without reification) at BudHac.

cheers,

Luite

[1]
https://github.com/ghcjs/ghcjs/blob/414eefb2bb8825b3c4c5cddfec4d79a142bc261a/src/Gen2/TH.hs
[2]
https://github.com/ghcjs/ghcjs-prim/blob/2dffdc2d732b044377037e1d6ebeac2812d4f9a4/GHCJS/Prim/TH/Eval.hs
[3]
https://github.com/ghcjs/ghcjs/blob/414eefb2bb8825b3c4c5cddfec4d79a142bc261a/lib/etc/thrunner.js
[4]
https://github.com/ghcjs/ghcjs-prim/blob/2dffdc2d732b044377037e1d6ebeac2812d4f9a4/GHCJS/Prim/TH/Types.hs#L29
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/ghc-devs/attachments/20140702/0be85ea5/attachment.html>