[GHC] #10871: Implement "fat" interface files which can be directly compiled without source

[GHC]

#10871: Implement "fat" interface files which can be directly compiled without
source
-------------------------------------+-------------------------------------
        Reporter:  ezyang            |                Owner:  ezyang
            Type:  feature request   |               Status:  new
        Priority:  normal            |            Milestone:
       Component:  Compiler          |              Version:  7.11
      Resolution:                    |             Keywords:  backpack
Operating System:  Unknown/Multiple  |         Architecture:
                                     |  Unknown/Multiple
 Type of failure:  None/Unknown      |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):
       Wiki Page:                    |
-------------------------------------+-------------------------------------

Comment (by simonmar):

 > I think the difference in opinion stems from whether or not it's
 reasonable to ask the user to have built the non-leaves prior to actually
 building the unit in question. In the case that it is necessary, the only
 way the user is going to be able to do this in practice is with external
 tooling, e.g. cabal-install. But this was true for pre-Backpack packages
 too; no one really can install a package without help from Cabal. And this
 seems to be fine.

 I'm not completely sure I understand what "built on the non-leaves" means,
 but I expect all of the packages (and instantiations thereof) that are
 dependencies of the current set of modules being built, are already built
 and installed in the package database.  Which as you say, is exactly as it
 is now.

 > But there are other things for which it's not reasonable to ask the user
 to have run Cabal beforehand. For example, I can work on a multi-module
 Haskell program without using Cabal, and it would pretty horrible user-
 experience if I had to "install" every module before I could use it in
 another program.

 Absolutely.  I expect to be able to do `ghc --make` on a collection of
 modules and signatures in a Backpack world, and provided all the external
 dependencies are satisfied in the package database, it should just work.

 > For example: today, if containers publishes Data.Map which has a map
 data type, I can use it immediately in an hs file, no fussing about with
 Cabal. If we translate this to Backpack, we get a data-map Backpack unit,
 which has a hole for the type of keys. It would be bad user experience to
 force a user to use Cabal to build and install all the instantiated copies
 of data-map they might want to use before they can build their
 application. To add insult to injury, if the key type in question is in
 the package I was working on, they would have separate it out into new
 unit containing just their type definition, so that could be installed
 first and the used to instantiated data-map, before they can go ahead and
 build the package they're thinking about.

 Yes - I do believe that to instantiate `Data.Map` you should have to build
 it (from the original source package) and install the instantiated result
 in the package database.  But our tooling is going to do this for us
 automatically.  This story is simple, fits with our current compilation
 model, and retains the logical separation between GHC (for building a
 collection of modules) and Cabal (for building and installing packages).

 I agree that this might be annoying from a user perspective if they aren't
 using the higher level tool, so I can see why you wanted to solve the
 problem differently.  But isn't it going to be strange if building my
 program also requires a long sequence of compilations of external modules?
 And where do the results go?  GHC has no business installing stuff, so
 they have to go in some local sandbox-like setup, which means that in some
 other project when you want the same instantiation you don't get to reuse
 the results.  This is exactly what the package database is for!  It caches
 compositions of packages.

 I think this relates to what you said earlier:

 > Here is where you might object, since I am cheating here, but what seems
 easiest is to have GHC and Cabal (but not cabal-install) conspire to
 deduplicate instantiated units "behind the scenes".

 I'm not sure exactly what mechanism you have in mind, but if Cabal and GHC
 have to conspire, what happens when you're using GHC by itself?  This does
 seem extremely murky to me.  Why not just use the package database, and
 have Cabal manage the installation of instantiated packages?

 So while I agree with you that it's important to have a story for using
 GHC(i) directly, in practice this isn't going to be the way people want to
 use it, and furthermore the more we try to make this a smooth user
 experience, the more we will end up putting features that should be in the
 higher-level tools into GHC itself, leading to a mess.  So I think we
 should retain the current clear division of labour: stack/cabal know about
 collections of packages, Cabal-the-library knows about building +
 installing single packages, and GHC knows about building collections of
 modules.

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/10871#comment:21>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler