Require -fexternal-interpreter support for future TH changes?

[Simon Marlow]

On 22 June 2016 at 11:37, Simon Peyton Jones <simonpj at microsoft.com> wrote:

> I’m ok with this.   *It would certainly be great not to support TWO
> mechanisms indefinitely*.
>
>
>
> What are the disadvantages to committing to this path?  Would anyone even
> notice?
>

Yes, people who are making changes to TH will need to ensure that their
changes work with -fexternal-interpreter.  In some cases that might mean
extra work, e.g. if we do TemplateHaskell/Introspective
<https://ghc.haskell.org/trac/ghc/wiki/TemplateHaskell/Introspective> then
potentially the whole of HsSyn needs to be binary serializable.  That
worries me quite a lot - THSyn is big but tractable, Generic deriving
handled the generation of the binary instances easily enough, but HsSyn is
another matter entirely.


> There are a lot of moving parts to the implementation, and I for one am
> utterly ignorant of how it all works.  I would love to see an
> implementation overview, either somewhere in the code or on a wiki page.
> Things like:
>
> ·         How, when, and where in the compiler is the separate process
> started?
>
> ·         How do the compiler and server communicate?  Unix pipes? Is it
> the same on Windows and Unix?
>
> ·         What is serialised, when, and how?  For example, GHC has some
> TH code to run.  Do we send a syntax tree?  Or compile to bytecode and send
> that?  Or what?
>
> ·         How are external references managed.  E.g. if the code to be
> run refers to ‘map’ I’m sure we don’t serialise the code for ‘map’.
>
> I’m sure there is a lot more. E.g the [wiki:RemoteGHCi wiki page] refers
> to “a library implementing a message type…” but I don’t know what that
> library is called or where it lives.
>

Yes, we should really have a page in the commentary with an overview of the
pieces and the main implementation strategy. I'll write one.

Cheers
Simon


>
>
> Thanks
>
>
>
> Simon
>
>
>
> *From:* ghc-devs [mailto:ghc-devs-bounces at haskell.org] *On Behalf Of *Simon
> Marlow
> *Sent:* 22 June 2016 09:51
> *To:* ghc-devs at haskell.org
> *Subject:* Require -fexternal-interpreter support for future TH changes?
>
>
>
> *Background*
>
> A few months ago I added -fexternal-interpreter to GHC:
>
>    - docs:
>    http://downloads.haskell.org/~ghc/latest/docs/html/users_guide/ghci.html#ghc-flag--fexternal-interpreter
>    <https://na01.safelinks.protection.outlook.com/?url=http:%2f%2fdownloads.haskell.org%2f~ghc%2flatest%2fdocs%2fhtml%2fusers_guide%2fghci.html%23ghc-flag--fexternal-interpreter&data=01%7C01%7Csimonpj%40064d.mgd.microsoft.com%7C5ca29759b5c9483e03fb08d39a7a60ea%7C72f988bf86f141af91ab2d7cd011db47%7C1&sdata=6t6FA3nUkP5FOi%2fqCBSr3GxMH2rwWNk89je1qBH9GfI%3d>
>    - wiki, rationale: https://ghc.haskell.org/trac/ghc/wiki/RemoteGHCi
>
> When -fexternal-interpreter is used, GHC runs interpreted code in a
> separate subprocess, and communicates with it using binary messages over a
> pipe.
>
> -fexternal-interpreter currently implements all of TH, quasi-quoting,
> annotations, and all the GHCi features except for some features of the
> debugger.  It is also now implemented on Windows, thanks to Tamar Christina.
>
> *Proposal*
>
> I'd like to propose that going forward we commit to maintaining full
> support for -fexternal-interpreter, with a view to making it the default.
>
> Why?
>
>    - -fexternal-interpreter will be a prerequisite for GHCJS support, so
>    maintaining full support for TH in -fexternal-interpreter will ensure that
>    everything that works with GHC works with GHCJS.
>    - We will be able to make simplifications in GHC and the build system
>    once -fexternal-interpreter is the default, because when compiling with
>    -prof or -dynamic we won't have to compile things twice any more.
>    - Ultimately we don't want to have two ways of doing everything,
>    because that's harder to maintain.
>
> How?
>
>    - I'll make all the TH and quasi-quoting tests run with and without
>    -fexternal-interpreter, so it will break validate if one of these fails.
>
> *Why now?*
>
> There are some TH changes in the pipeline that will need special attention
> to work with -fexternal-interpreter.  e.g.
> https://phabricator.haskell.org/D2286 and
> https://ghc.haskell.org/trac/ghc/wiki/TemplateHaskell/Introspective, so
> I'd like to raise it now so we can keep the issue in mind.
>
>
>
> Cheers
>
> Simon
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20160622/b79ceb17/attachment.html>