[GHC] #10871: Implement "fat" interface files which can be directly compiled without source

[GHC]

#10871: Implement "fat" interface files which can be directly compiled without
source
-------------------------------------+-------------------------------------
        Reporter:  ezyang            |                Owner:  ezyang
            Type:  feature request   |               Status:  new
        Priority:  normal            |            Milestone:
       Component:  Compiler          |              Version:  7.11
      Resolution:                    |             Keywords:  backpack
Operating System:  Unknown/Multiple  |         Architecture:
                                     |  Unknown/Multiple
 Type of failure:  None/Unknown      |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):
       Wiki Page:                    |
-------------------------------------+-------------------------------------

Comment (by ezyang):

 I would agree that the primary payoff of this feature is Backpack; without
 it, I think there is much less motivation. I think the "external core"
 use-case could be quite compelling, but there's more work to do extracting
 the interface file parser into its own library.

 Do fat interfaces violate cabal install "depending on its inputs, and
 nothing more"? When I try to answer this question, I find myself asking,
 "what distinguishes these from normal interface files which we save for
 packages that are already installed?" Let me recap Duncan's new plan for
 implementing cabal install so that it is minimally affected by the
 existing database of packages:

 1. We first solve dependencies, without making any reference to the
 existing package database. This allows us to compute a number of IPIDs of
 configured packages.
 2. Then, we *improve* these configured packages into installed packages
 simply by looking up their IPIDs in the package database.
 3. When we compile, we begin as if we had already compiled the already
 installed packages, and finish up building and installing the rest of the
 packages in the plan.

 The emphasis seems to be removing side effects from step (1); but of
 course we do want to avoid rebuilding things that are already built, so
 (2) is "effectful" but in a benign way.

 I think this plan, which avoids side-effects during dependency resolution,
 works for Backpack fat interfaces too. Here's how it looks:

 1. Once again, we solve dependencies. This gives a number of component IDs
 for all configured components.
 2. We improve these configured components into installed components. Some
 of these will map to pre-compiled components, but others will map to
 *indefinite components* which were installed with fat interface file
 3. When we compile, we begin as if we had compiled all the installed
 packages, and partially compiled all the indefinite ones, and just
 "finish" up the rest of the build according to the plan.

 There is extra complexity with ensuring that instantiated units get
 deduplicated correctly (arising from Backpack's applicativity) but I
 really think that fat interface files support the method that you are
 looking for.

 Let me also point out the other big difference between fat interface files
 and source files: GHC *always* knows how to compile fat interface files.
 You cannot have any sort of custom build system associated with them.
 They're just a big pile of inlinings; there's no external C code, no
 preprocessing, etc. This means, for example, that if you want a indefinite
 component with some C code, the C code gets built ONCE when you initially
 installed the indefinite package, not each time you instantiate the
 component.

 I'd really like to get you on board with this new plan, so let me know if
 you want to chat about it some more.

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/10871#comment:10>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler