GHCJS now runs Template Haskell on node.js - Any interest in out of process TH for general cross compilation?

[Luite Stegeman]

I think GHC could use more or less the same communication method as GHCJS
now does: Start some user-specifiied process and send messages through
pipes (GHCJS uses stdin/stderr of the node process), with the difference
that it would get dynamic libraries for the target rather than blobs of JS
code. That user process is then responsible for setting up the actual
communication with the runner on the emulator or development device.

A requirement for complete TH support is that more code can be loaded at
runtime, so that multiple splices can be run by the same runner (because of
the persistent map, qGetQ / qPutQ), I'm not sure if this is problematic on
iOS.



On Thu, Jul 3, 2014 at 2:54 AM, Carter Schonwald <carter.schonwald at gmail.com
> wrote:

> This would probably be a great boon for those trying to use haskell for
> Android and IOS right? how might the emulation setup work for those?
>
>
>
>
> On Wed, Jul 2, 2014 at 2:20 PM, Carter Schonwald <
> carter.schonwald at gmail.com> wrote:
>
>> wow, this is great work!
>>
>> If theres a clear path to getting the generic tooling into 7.10, i'm all
>> for it :) (and willing to help on concrete mechanical subtasks)
>>
>>
>> On Wed, Jul 2, 2014 at 12:14 PM, Luite Stegeman <stegeman at gmail.com>
>> wrote:
>>
>>> hi all,
>>>
>>> I've added some code [1] [2] to GHCJS to make it run Template Haskell
>>> code on node.js, rather than using the GHC linker. GHCJS has supported TH
>>> for a long time now, but so far always relied on native (host) code for it.
>>> This is the main reason that GHCJS always builds native and JavaScript code
>>> for everything (another is that Cabal Setup.hs scripts need to be compiled
>>> to some host-runnable form, but that can also be JavaScript if you have
>>> node.js)
>>>
>>> Now besides the compiler having to do twice the work, this has some
>>> other disadvantages:
>>>
>>> - Our JavaScript code has the same dependencies (packages) as native
>>> code, which means packages like unix or Win32 show up somewhere, depending
>>> on the host environment. This also limits our options in choosing
>>> JS-specific packages.
>>> - The Template Haskell code runs on the host environment, which might be
>>> slightly different from the target, for example in integer size or
>>> operating system specific constants.
>>>
>>> Moreover, building native code made the GHCJS installation procedure
>>> more tricky, making end users think about libgmp or libiconv locations,
>>> since it basically required the same preparation as building GHC from
>>> source. This change will make installing much easier and more reliable (we
>>> still have to update the build scripts).
>>>
>>> How it works is pretty simple:
>>>
>>> - When any code needs to be run on the target (hscCompileCoreExpr,
>>> through the Hooks API new in GHC 7.8), GHCJS starts a node.js process with
>>> the thrunner.js [3] script,
>>> - GHCJS sends its RTS and the Template Haskell server code [1] to
>>> node.js, the script starts a Haskell thread running the server,
>>> - for every splice, GHCJS compiles it to JavaScript and links it using
>>> its incremental linking functionality. The code for the splice, including
>>> dependencies that have not yet been sent to the runner (for earlier
>>> splices), is then sent in a RunTH [4] message,
>>> - the runner loads and runs the code in the Q monad, can send queries to
>>> GHCJS for reification,
>>> - the runner sends back the result as a serialized Template Haskell AST
>>> (using GHC.Generics for the Binary instances).
>>>
>>> All Template Haskell functionality is supported, including recent
>>> additions for reifying modules and annotations. I still need to clean up
>>> and push the patches for the directory and process packages, but after
>>> that, the TH code can read/write files, run processes and interact with
>>> them and make network connections, all through node.js.
>>>
>>> Now since this approach is in no way specific to JavaScript, I was
>>> wondering if there's any interest in getting this functionality into GHC
>>> 7.10 for general cross compilation. The runner would be a native (target)
>>> program with dynamic libraries (or object files) being sent over to the
>>> target machine (or emulator) for the splices.
>>>
>>> Thanks to Andras Slemmer from Prezi who helped build the initial proof
>>> of concept (without reification) at BudHac.
>>>
>>> cheers,
>>>
>>> Luite
>>>
>>> [1]
>>> https://github.com/ghcjs/ghcjs/blob/414eefb2bb8825b3c4c5cddfec4d79a142bc261a/src/Gen2/TH.hs
>>> [2]
>>> https://github.com/ghcjs/ghcjs-prim/blob/2dffdc2d732b044377037e1d6ebeac2812d4f9a4/GHCJS/Prim/TH/Eval.hs
>>> [3]
>>> https://github.com/ghcjs/ghcjs/blob/414eefb2bb8825b3c4c5cddfec4d79a142bc261a/lib/etc/thrunner.js
>>> [4]
>>> https://github.com/ghcjs/ghcjs-prim/blob/2dffdc2d732b044377037e1d6ebeac2812d4f9a4/GHCJS/Prim/TH/Types.hs#L29
>>>
>>> _______________________________________________
>>> Glasgow-haskell-users mailing list
>>> Glasgow-haskell-users at haskell.org
>>> http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/ghc-devs/attachments/20140703/b8c1bd73/attachment.html>