[GHC] #11244: Compiler plugins should not have visibility controlled by -package

[GHC]

#11244: Compiler plugins should not have visibility controlled by -package
-------------------------------------+-------------------------------------
        Reporter:  ezyang            |                Owner:  ezyang
            Type:  bug               |               Status:  new
        Priority:  normal            |            Milestone:  8.0.1
       Component:  Compiler          |              Version:  7.11
      Resolution:                    |             Keywords:
Operating System:  Unknown/Multiple  |         Architecture:
                                     |  Unknown/Multiple
 Type of failure:  None/Unknown      |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):
       Wiki Page:                    |
-------------------------------------+-------------------------------------
Description changed by ezyang:

Old description:

> The current way compiler plugins are recommended to be specified is by
> putting it in a package and then referring to the exported module name in
> the `-fplugin` flag.  Consequently, the recommended way to use a plugin
> from Cabal is to put it in `build-depends`. For example,
> [https://hackage.haskell.org/package/ghc-typelits-natnormalise ghc-
> typelits-natnormalise] is used by [http://hackage.haskell.org/package
> /clash-prelude clash-prelude], whose [http://hackage.haskell.org/package
> /clash-prelude-0.10.4/clash-prelude.cabal build-depends] depends on it
> explicitly.
>
> There are numerous downsides to operating in this manner:
> 1. Cabal will always link in any library which is `build-depend`ed on,
> even if there is no runtime dependency on the package in question. For
> example, suppose the default `cabal-init` package with a `build-depends:
> ghc` and doesn't use it, you get:
> {{{
> ezyang at sabre:~/Dev/labs/link$ cat Main.hs
> module Main where
>
> main :: IO ()
> main = putStrLn "Hello, Haskell!"
> ezyang at sabre:~/Dev/labs/link$ cabal configure --enable-executable-
> dynamic; cabal build
> # elided
> ezyang at sabre:~/Dev/labs/link$ ldd dist/build/link/link | grep libHSghc
>         libHSghc-7.10.2-JzwEp1oQ8kA7NFNTGk1ho5-ghc7.10.2.so =>
> /srv/code/ghc-7.10.2/usr/lib/ghc-7.10.2/ghc_JzwEp1oQ8kA7NFNTGk1ho5/libHSghc-7.10.2-JzwEp1oQ8kA7NFNTGk1ho5-ghc7.10.2.so
> (0x00007f591f47a000)
> }}}
>    That's terrible. And it's inflicted on every package which depends on
> a library that uses the plugin. The situation is better with static
> linking, since the linker can drop unreferenced object files; but we
> shouldn't be passing it to the linker in the first place.
> 2. It should be possible use a compiler plugin with a cross-compiler
> (e.g. GHCJS, etc.); unlike Template Haskell, the plugin must be written
> with the host compiler in mind. But the package database contains
> libraries compiled in the //target language//; whereas a compiler plugin
> must be compiled in the //host language//.
>
> It seems clear that using `-package` to bring a plugin into scope is
> highly undesirable. So what should we do?  The necessary information to
> load a plugin is:
>
> 1. The package DB entry for the package it lives in (and information
> about all its transitive dependencies, because we may need to load code
> from them as well),
> 2. The actual module which contains the plugin.
>
> This leaves us with a number of options on the GHC end:
>
> 1. We could have a explicitly, IPID qualified method of specifying
> plugins.  So, instead of saying `-fplugin ModuleName` and being at the
> mercy of what is exposed, you could say `-fplugin
> foo-0.1-xxxx:ModuleName` and as long as that package is usable (not
> ignored/broken) it would get loaded.
>
> 2. We could have a *completely separate* package database for plugins.
> You would control it using `-plugin-package` rather than `-package`; you
> don't even have to use the default system/user databases (which means
> that it might be possible to support a cross-compiler, although the
> details are fuzzy in my head). A benefit of this approach is that you can
> support the `-fplugin MyModule` mode of use, and have Cabal feed in the
> extra plugin package arguments to give meaning to this expression.
>
>    An alternate command line interface (suggested by SPJ) would be that
> to load a plugin, you specify a path to the package database, package,
> and module in question, something like `-fplugin
> /path/to/package.conf/foo-0.1-xxxx.conf:MyPlugin` (one difficulty with
> this approach is that you may need multiple package databases to get this
> to work.)
>
> There would also be necessary Cabal adjustments, to get people to not put
> their plugins in `build-depends`, but a different dependency section, and
> how to have Cabal feed the IPID of the plugin to GHC.
>
> I'm happy to implement, but I'm looking for some consensus on what to do.
> (1) is easier to implement, but I dare say harder to integrate with
> Cabal.

New description:

 The current way compiler plugins are recommended to be specified is by
 putting it in a package and then referring to the exported module name in
 the `-fplugin` flag.  Consequently, the recommended way to use a plugin
 from Cabal is to put it in `build-depends`. For example,
 [https://hackage.haskell.org/package/ghc-typelits-natnormalise ghc-
 typelits-natnormalise] is used by [http://hackage.haskell.org/package
 /clash-prelude clash-prelude], whose [http://hackage.haskell.org/package
 /clash-prelude-0.10.4/clash-prelude.cabal build-depends] depends on it
 explicitly.

 There are numerous downsides to operating in this manner:
 1. Cabal will always link in any library which is `build-depend`ed on,
 even if there is no runtime dependency on the package in question. For
 example, suppose the default `cabal-init` package with a `build-depends:
 ghc` and doesn't use it, you get:
 {{{
 ezyang at sabre:~/Dev/labs/link$ cat Main.hs
 module Main where

 main :: IO ()
 main = putStrLn "Hello, Haskell!"
 ezyang at sabre:~/Dev/labs/link$ cabal configure --enable-executable-dynamic;
 cabal build
 # elided
 ezyang at sabre:~/Dev/labs/link$ ldd dist/build/link/link | grep libHSghc
         libHSghc-7.10.2-JzwEp1oQ8kA7NFNTGk1ho5-ghc7.10.2.so =>
 /srv/code/ghc-7.10.2/usr/lib/ghc-7.10.2/ghc_JzwEp1oQ8kA7NFNTGk1ho5/libHSghc-7.10.2-JzwEp1oQ8kA7NFNTGk1ho5-ghc7.10.2.so
 (0x00007f591f47a000)
 }}}
    That's terrible. And it's inflicted on every package which depends on a
 library that uses the plugin. The situation is better with static linking,
 since the linker can drop unreferenced object files; but we shouldn't be
 passing it to the linker in the first place.
 2. It should be possible use a compiler plugin with a cross-compiler (e.g.
 GHCJS, etc.); unlike Template Haskell, the plugin must be written with the
 host compiler in mind. But the package database contains libraries
 compiled in the //target language//; whereas a compiler plugin must be
 compiled in the //host language//.

 It seems clear that using `-package` to bring a plugin into scope is
 highly undesirable. So what should we do?  The necessary information to
 load a plugin is:

 1. The package DB entry for the package it lives in (and information about
 all its transitive dependencies, because we may need to load code from
 them as well),
 2. The actual module which contains the plugin.

 This leaves us with a number of options on the GHC end:

 1. We could have a explicitly, IPID qualified method of specifying
 plugins.  So, instead of saying `-fplugin ModuleName` and being at the
 mercy of what is exposed, you could say `-fplugin foo-0.1-xxxx:ModuleName`
 and as long as that package is usable (not ignored/broken) it would get
 loaded.

 2. We could have a *completely separate* package database for plugins.
 You would control it using `-plugin-package` rather than `-package`; you
 don't even have to use the default system/user databases (which means that
 it might be possible to support a cross-compiler, although the details are
 fuzzy in my head). A benefit of this approach is that you can support the
 `-fplugin MyModule` mode of use, and have Cabal feed in the extra plugin
 package arguments to give meaning to this expression.

    An alternate command line interface (suggested by SPJ) would be that to
 load a plugin, you specify a path to the package database, package, and
 module in question, something like `-fplugin
 /path/to/package.conf/foo-0.1-xxxx.conf:MyPlugin` (one difficulty with
 this approach is that you may need multiple package databases to get this
 to work.)

 3. We could use the same databases, but a *separate* package namespace.
 So I can say `ghc -hide-all-packages -plugin-package-id foo-0.1-xxx`; the
 `-plugin-package-id` makes it so that I can say `-fplugin FooPlugin` later
 in the command line options.  This is desirable because it allows you to
 continue to say `{-# OPTIONS -fplugin FooPlugin #-}` in a Haskell file,
 turning on the plugin per-module, while letting Cabal configure where it
 points to.

 There would also be necessary Cabal adjustments, to get people to not put
 their plugins in `build-depends`, but a different dependency section, and
 how to have Cabal feed the IPID of the plugin to GHC.

 I'm happy to implement, but I'm looking for some consensus on what to do.

--

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/11244#comment:2>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler