Patch/feature proposal: "Source plugins"

Thomas Schilling nominolo at googlemail.com
Wed Jul 31 01:30:01 CEST 2013


I added my (very rough) take on this issue at: https://github.com/nominolo/ghc-phase-plugins

I wanted to implement this outside of GHC, so I had to copy HscMain and DriverPipeline.

The key part is the FileHooks datatype:

  type Hook a = a -> a

  data FileHooks = FileHooks
    { hookParse :: Hook (ModSummary -> HHsc HsParsedModule)
    , hookTypecheckRename :: Hook (HscEnv -> ModSummary -> HsParsedModule 
                                          -> HHsc TcGblEnv)
    , hookDesugar :: Hook (HscEnv -> ModSummary -> TcGblEnv -> HHsc ModGuts)
    , hookOptimize :: Hook (ModGuts -> HHsc ModGuts)
    , hookWriteIface :: Hook (ModIface -> Bool -> ModSummary -> HHsc ())
    , hookCodeGen :: Hook (ModIface -> ModDetails -> CgGuts -> ModSummary
                                    -> HHsc (Maybe FilePath))
    , hookPostBackendPhase :: Hook (DynFlags -> HscSource -> HscTarget -> Phase)
    -- there are probably more things we want to add here
    }

The basic idea is simple: each significant component of the compiler can be customised by choosing the right hook implementation.  Each hook gets the default implementation as its first argument and can choose whatever it wants to do with it.  Where the current code would call, say, "hscOptimize guts", it now calls "hookOptimize hooks hscOptimize guts".

The messy bit is that it's up to the hook to establish the necessary (often undocumented) invariants for the result.  A cleaner API would probably require larger refactorings and a more careful design of the high-level pipeline API, though.

The HHsc type is currently just "IO (Messages, Maybe a)", but perhaps it's better to just export the Hsc monad currently used internally in HscMain.

Edsko's design, which seems to focus on type checker/quasi-quoting phases, could be included in this.  In fact it would be nice if we could actually untangle the renamed and type checker again. A few years back I discussed this with SPJ, and we concluded that it should be possible to do that.

There are two immediate use cases I have in mind for this design:

 1. IDE tools which run custom analyses in between compilation.  A means to omit certain phases is also useful.  (Right now, specifying -fno-code also skips desugaring which means you lose warnings regarding overlapping and missing pattern matches.)

 2. Custom backends.  There are now a number of projects that implement custom backends for GHC (ghcjs, Lambdachine).  Being able to reuse all of the front-end and plug in a custom code-generator is very useful.

 / Thomas


On 25 Jul 2013, at 17:12, Edsko de Vries <edskodevries at gmail.com> wrote:

> Ok, I have now finally got managed to compile and test my code against
> ghc head. I have created a wiki page for the proposal
> 
> http://ghc.haskell.org/trac/ghc/wiki/FrontendPluginsProposal
> 
> and have attached patches against both 7.4.2 and against HEAD.
> Hopefully this is now a good starting point for a design that works
> for everybody.
> 
> -E
> 
> On Mon, Jul 8, 2013 at 10:46 PM, Simon Peyton-Jones
> <simonpj at microsoft.com> wrote:
>> Edsko, Luite, Thomas, and others
>> 
>> 
>> 
>> Thanks.  I wonder whether you might write a Wiki page saying
>> 
>> ·        exactly what you want to achieve, with examples, and
>> 
>> ·        sketching how you hope to achieve it
>> 
>> 
>> 
>> You may want to do a few rounds on that among yourselves; perhaps you have
>> various different goals in mind that we can address at once.
>> 
>> 
>> 
>> Selfishly, I don’t really want to review code until I have a very clear
>> picture of what it is trying to achieve.   (And I’m rushing towards the POPL
>> deadline anyway!)
>> 
>> 
>> 
>> Simon
>> 
>> 
>> 
>> From: Edsko de Vries [mailto:edskodevries at gmail.com]
>> Sent: 08 July 2013 18:02
>> To: Simon Peyton-Jones
>> Cc: Luite Stegeman; Thomas Schilling; ghc-devs at haskell.org; Nicolas Frisby
>> 
>> 
>> Subject: Re: Patch/feature proposal: "Source plugins"
>> 
>> 
>> 
>> Ok, sorry all for the delay. Attached is a "frontend plugins" patch for ghc
>> 7.4.2. It will not work for ghc HEAD because the structure of the compiler
>> has changed in various places, but I cannot currently compile my code
>> against HEAD. Making this available now because it's been far too long
>> already :) Comments/suggestions/feedback welcome. Hopefully the patch is
>> pretty self-explanatory; and it's only small.
>> 
>> 
>> 
>> -E
>> 
>> 
>> 
>> On Mon, Jul 1, 2013 at 9:32 AM, Edsko de Vries <edskodevries at gmail.com>
>> wrote:
>> 
>> Yes, I fully intend to create a ticket with a detailed description and a
>> first patch, but I've been struggling with the latest HEAD, and specifically
>> the fact that it now uses dynamic libraries for TH. I (think I am) stuck at
>> a Cabal bug and I cannot currently build my code at all :-/ Duncan is
>> looking into this though.
>> 
>> 
>> 
>> Edsko
>> 
>> 
>> 
>> On Fri, Jun 28, 2013 at 1:43 PM, Simon Peyton-Jones <simonpj at microsoft.com>
>> wrote:
>> 
>> I’m confused as to details here.
>> 
>> ·         Edsko is doing something; Nick is doing something else (attached
>> for completeness).
>> 
>> ·         I can’t locate a Trac Wiki page that describes the design
>> 
>> 
>> 
>> I’m more than happy to adopt patches that improve the plugin API, but you’ll
>> have to lead me through it!
>> 
>> 
>> 
>> No hurry, just when you are ready.
>> 
>> 
>> 
>> Simon
>> 
>> 
>> 
>> From: Edsko de Vries [mailto:edskodevries at gmail.com]
>> Sent: 26 June 2013 09:21
>> To: Luite Stegeman
>> Cc: Simon Peyton-Jones; Thomas Schilling; ghc-devs at haskell.org
>> 
>> 
>> Subject: Re: Patch/feature proposal: "Source plugins"
>> 
>> 
>> 
>> Hi Luite,
>> 
>> 
>> 
>> I was fully planning on a first version of the patch yesterday, but so far
>> my efforts were thwarted by annoying problems with dynamic libraries (not --
>> directly -- related to the patch at all). I will try again today :)
>> 
>> 
>> 
>> Edsko
>> 
>> 
>> 
>> On Wed, Jun 26, 2013 at 8:51 AM, Luite Stegeman <stegeman at gmail.com> wrote:
>> 
>> Any news on this? I'd really like to have this in GHC 7.8.1 so that we can
>> release a fully working GHCJS with GhcMake functionality based on it. I'd be
>> happy to help write the patch.
>> 
>> 
>> 
>> luite
>> 
>> 
>> 
>> 
>> 
>> On Tue, Jun 11, 2013 at 3:21 PM, Simon Peyton-Jones <simonpj at microsoft.com>
>> wrote:
>> 
>> Guys,
>> 
>> I'm not following the details here, but I'm open to suggestions (patches,
>> even) that improve the GHC API.
>> 
>> Simon
>> 
>> 
>> | -----Original Message-----
>> | From: ghc-devs-bounces at haskell.org [mailto:ghc-devs-bounces at haskell.org]
>> | On Behalf Of Thomas Schilling
>> | Sent: 11 June 2013 12:53
>> | To: Edsko de Vries
>> | Cc: ghc-devs at haskell.org
>> | Subject: Re: Patch/feature proposal: "Source plugins"
>> |
>> | On 5 June 2013 13:51, Edsko de Vries <edskodevries at gmail.com> wrote:
>> | > It is a little bit messy mostly because parts of the AST get lost
>> | along the
>> | > way: quasi-quotes in the renamer, data type declarations and other
>> | things
>> | > during type checking. A more ideal way, but also more time consuming,
>> | would
>> | > be to change this so that the renamer leaves evidence of the quasi-
>> | quotes in
>> | > the tree, and the type checker returns the entire tree type checked,
>> | rather
>> | > than just a subset. I think that ultimately this is the better
>> | approach, at
>> | > least for our purposes -- I'm not sure about other tools, but since
>> | this
>> | > would be a larger change that affects larger parts of the ghc pipeline
>> | I'm
>> | > not sure that I'll be able to do it.
>> |
>> | I needed something similar.  In particular, I built a custom code
>> | generator, but now I need a similar feature for extracting information
>> | from a Haskell file (for IDE features).
>> |
>> | Since I needed to modify one-shot compilation mode I couldn't use the
>> | GHC API.  For the IDE stuff I'm using Shake as the build manager, so
>> | that also needs a customized one-shot mode.  For my current
>> | implementation I just copied and adapted the necessary parts of
>> | HscMain, DriverPipeline, etc.  That's very messy, fragile and breaks
>> | on every GHC release so I'd really like to see the necessary features
>> | put into GHC.
>> |
>> | Do you have a working patch somewhere?
>> |
>> | _______________________________________________
>> | ghc-devs mailing list
>> | ghc-devs at haskell.org
>> | http://www.haskell.org/mailman/listinfo/ghc-devs
>> 
>> _______________________________________________
>> ghc-devs mailing list
>> ghc-devs at haskell.org
>> http://www.haskell.org/mailman/listinfo/ghc-devs
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 203 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://www.haskell.org/pipermail/ghc-devs/attachments/20130731/ae92173e/attachment.pgp>


More information about the ghc-devs mailing list