Require -fexternal-interpreter support for future TH changes?

[Simon Peyton Jones]

I’m ok with this.   It would certainly be great not to support TWO mechanisms indefinitely.

What are the disadvantages to committing to this path?  Would anyone even notice?

There are a lot of moving parts to the implementation, and I for one am utterly ignorant of how it all works.  I would love to see an implementation overview, either somewhere in the code or on a wiki page.  Things like:

·         How, when, and where in the compiler is the separate process started?

·         How do the compiler and server communicate?  Unix pipes? Is it the same on Windows and Unix?

·         What is serialised, when, and how?  For example, GHC has some TH code to run.  Do we send a syntax tree?  Or compile to bytecode and send that?  Or what?

·         How are external references managed.  E.g. if the code to be run refers to ‘map’ I’m sure we don’t serialise the code for ‘map’.
I’m sure there is a lot more. E.g the [wiki:RemoteGHCi wiki page] refers to “a library implementing a message type…” but I don’t know what that library is called or where it lives.

Thanks

Simon

From: ghc-devs [mailto:ghc-devs-bounces at haskell.org] On Behalf Of Simon Marlow
Sent: 22 June 2016 09:51
To: ghc-devs at haskell.org
Subject: Require -fexternal-interpreter support for future TH changes?

Background
A few months ago I added -fexternal-interpreter to GHC:

  *   docs: http://downloads.haskell.org/~ghc/latest/docs/html/users_guide/ghci.html#ghc-flag--fexternal-interpreter<https://na01.safelinks.protection.outlook.com/?url=http:%2f%2fdownloads.haskell.org%2f~ghc%2flatest%2fdocs%2fhtml%2fusers_guide%2fghci.html%23ghc-flag--fexternal-interpreter&data=01%7C01%7Csimonpj%40064d.mgd.microsoft.com%7C5ca29759b5c9483e03fb08d39a7a60ea%7C72f988bf86f141af91ab2d7cd011db47%7C1&sdata=6t6FA3nUkP5FOi%2fqCBSr3GxMH2rwWNk89je1qBH9GfI%3d>
  *   wiki, rationale: https://ghc.haskell.org/trac/ghc/wiki/RemoteGHCi
When -fexternal-interpreter is used, GHC runs interpreted code in a separate subprocess, and communicates with it using binary messages over a pipe.
-fexternal-interpreter currently implements all of TH, quasi-quoting, annotations, and all the GHCi features except for some features of the debugger.  It is also now implemented on Windows, thanks to Tamar Christina.
Proposal
I'd like to propose that going forward we commit to maintaining full support for -fexternal-interpreter, with a view to making it the default.
Why?

  *   -fexternal-interpreter will be a prerequisite for GHCJS support, so maintaining full support for TH in -fexternal-interpreter will ensure that everything that works with GHC works with GHCJS.
  *   We will be able to make simplifications in GHC and the build system once -fexternal-interpreter is the default, because when compiling with -prof or -dynamic we won't have to compile things twice any more.
  *   Ultimately we don't want to have two ways of doing everything, because that's harder to maintain.
How?

  *   I'll make all the TH and quasi-quoting tests run with and without -fexternal-interpreter, so it will break validate if one of these fails.

Why now?

There are some TH changes in the pipeline that will need special attention to work with -fexternal-interpreter.  e.g. https://phabricator.haskell.org/D2286 and https://ghc.haskell.org/trac/ghc/wiki/TemplateHaskell/Introspective, so I'd like to raise it now so we can keep the issue in mind.



Cheers

Simon
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20160622/e50a124a/attachment-0001.html>