Require -fexternal-interpreter support for future TH changes?

Wed Jun 22 11:31:50 UTC 2016

How's this?
https://ghc.haskell.org/trac/ghc/wiki/Commentary/Compiler/ExternalInterpreter

I don't want to go into too much detail in the wiki, because details are
more likely to stay current if they're in Notes in the code.  There are
already a few Notes (e.g.
https://phabricator.haskell.org/diffusion/GHC/browse/master/compiler/ghci/GHCi.hs;619958832cbe11096cae3dac9a0a7a5591163a00$86)
but if anything is confusing I'll happily add more Notes.

Cheers
Simon

On 22 June 2016 at 11:50, Simon Marlow <marlowsd at gmail.com> wrote:

> On 22 June 2016 at 11:37, Simon Peyton Jones <simonpj at microsoft.com>
> wrote:
>
>> I’m ok with this.   *It would certainly be great not to support TWO
>> mechanisms indefinitely*.
>>
>>
>>
>> What are the disadvantages to committing to this path?  Would anyone even
>> notice?
>>
>
> Yes, people who are making changes to TH will need to ensure that their
> changes work with -fexternal-interpreter.  In some cases that might mean
> extra work, e.g. if we do TemplateHaskell/Introspective
> <https://ghc.haskell.org/trac/ghc/wiki/TemplateHaskell/Introspective>
> then potentially the whole of HsSyn needs to be binary serializable.  That
> worries me quite a lot - THSyn is big but tractable, Generic deriving
> handled the generation of the binary instances easily enough, but HsSyn is
> another matter entirely.
>
>
>> There are a lot of moving parts to the implementation, and I for one am
>> utterly ignorant of how it all works.  I would love to see an
>> implementation overview, either somewhere in the code or on a wiki page.
>> Things like:
>>
>> ·         How, when, and where in the compiler is the separate process
>> started?
>>
>> ·         How do the compiler and server communicate?  Unix pipes? Is it
>> the same on Windows and Unix?
>>
>> ·         What is serialised, when, and how?  For example, GHC has some
>> TH code to run.  Do we send a syntax tree?  Or compile to bytecode and send
>> that?  Or what?
>>
>> ·         How are external references managed.  E.g. if the code to be
>> run refers to ‘map’ I’m sure we don’t serialise the code for ‘map’.
>>
>> I’m sure there is a lot more. E.g the [wiki:RemoteGHCi wiki page] refers
>> to “a library implementing a message type…” but I don’t know what that
>> library is called or where it lives.
>>
>
> Yes, we should really have a page in the commentary with an overview of
> the pieces and the main implementation strategy. I'll write one.
>
> Cheers
> Simon
>
>
>>
>>
>> Thanks
>>
>>
>>
>> Simon
>>
>>
>>
>> *From:* ghc-devs [mailto:ghc-devs-bounces at haskell.org] *On Behalf Of *Simon
>> Marlow
>> *Sent:* 22 June 2016 09:51
>> *To:* ghc-devs at haskell.org
>> *Subject:* Require -fexternal-interpreter support for future TH changes?
>>
>>
>>
>> *Background*
>>
>> A few months ago I added -fexternal-interpreter to GHC:
>>
>>    - docs:
>>    http://downloads.haskell.org/~ghc/latest/docs/html/users_guide/ghci.html#ghc-flag--fexternal-interpreter
>>    <https://na01.safelinks.protection.outlook.com/?url=http:%2f%2fdownloads.haskell.org%2f~ghc%2flatest%2fdocs%2fhtml%2fusers_guide%2fghci.html%23ghc-flag--fexternal-interpreter&data=01%7C01%7Csimonpj%40064d.mgd.microsoft.com%7C5ca29759b5c9483e03fb08d39a7a60ea%7C72f988bf86f141af91ab2d7cd011db47%7C1&sdata=6t6FA3nUkP5FOi%2fqCBSr3GxMH2rwWNk89je1qBH9GfI%3d>
>>    - wiki, rationale: https://ghc.haskell.org/trac/ghc/wiki/RemoteGHCi
>>
>> When -fexternal-interpreter is used, GHC runs interpreted code in a
>> separate subprocess, and communicates with it using binary messages over a
>> pipe.
>>
>> -fexternal-interpreter currently implements all of TH, quasi-quoting,
>> annotations, and all the GHCi features except for some features of the
>> debugger.  It is also now implemented on Windows, thanks to Tamar Christina.
>>
>> *Proposal*
>>
>> I'd like to propose that going forward we commit to maintaining full
>> support for -fexternal-interpreter, with a view to making it the default.
>>
>> Why?
>>
>>    - -fexternal-interpreter will be a prerequisite for GHCJS support, so
>>    maintaining full support for TH in -fexternal-interpreter will ensure that
>>    everything that works with GHC works with GHCJS.
>>    - We will be able to make simplifications in GHC and the build system
>>    once -fexternal-interpreter is the default, because when compiling with
>>    -prof or -dynamic we won't have to compile things twice any more.
>>    - Ultimately we don't want to have two ways of doing everything,
>>    because that's harder to maintain.
>>
>> How?
>>
>>    - I'll make all the TH and quasi-quoting tests run with and without
>>    -fexternal-interpreter, so it will break validate if one of these fails.
>>
>> *Why now?*
>>
>> There are some TH changes in the pipeline that will need special
>> attention to work with -fexternal-interpreter.  e.g.
>> https://phabricator.haskell.org/D2286 and
>> https://ghc.haskell.org/trac/ghc/wiki/TemplateHaskell/Introspective, so
>> I'd like to raise it now so we can keep the issue in mind.
>>
>>
>>
>> Cheers
>>
>> Simon
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20160622/5b9ddd0f/attachment-0001.html>