[GHC] #12518: Allow customizing immutable package dbs by stacking

[GHC]

#12518: Allow customizing immutable package dbs by stacking
-------------------------------------+-------------------------------------
        Reporter:  harendra          |                Owner:
            Type:  feature request   |               Status:  new
        Priority:  normal            |            Milestone:
       Component:  Package system    |              Version:  8.0.1
      Resolution:                    |             Keywords:
Operating System:  Unknown/Multiple  |         Architecture:
                                     |  Unknown/Multiple
 Type of failure:  None/Unknown      |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):
       Wiki Page:                    |
-------------------------------------+-------------------------------------
Description changed by harendra:

@@ -36,6 +36,19 @@
- option is used GHC will use `GHC_PACKAGE_PATH` or the -package-db options
- as a stack of dbs.  The first db in the path or the first CLI option will
- be considered the top of the stack. GHC will choose a package from the
- first db from the top of the stack irrespective of the version of the
- package. If the package is broken it should report error rather than
- silently choosing from the next db.
+ option is used GHC will use `GHC_PACKAGE_PATH` or the `-package-db`
+ options to specify a stack of dbs.  The first db in the path or the last
+ CLI option will be considered the top of the stack.
+
+ The default behavior of GHC is to union all databases whereas this feature
+ is about vertically stacking them. The following stacking rules will
+ apply:
+
+ * GHC will search a package from top to bottom and stop at the first db in
+ which the package exists.
+ * If GHC is not looking for a specific version then it will stop at the
+ first db in which ANY version is found.
+ * It will ignore the hidden packages when searching, but `-package <pkg>`
+ will have the effect of unhiding all versions of `<pkg>`.
+ * If there are multiple versions available in the same db then usual rules
+ of latest non-broken package will be used to select one.
+ * If the candidate package(s) found is broken it will not search further.
+ When a broken package is needed in compilation it will report an error. It
+ will not report errors in general when a broken package is encountered.

New description:

 = Package Selection Across Multiple Package DBs =

 As explained by ezyang, when there are multiple package dbs, GHC chooses
 the packages to use in the following manner. I might have misunderstood so
 please correct me if I am wrong.
 * If there are multiple packages with different versions the latest non-
 broken version is chosen.
 * If the there are multiple packages with the same version the behavior is
 unspecified.
 * If there are multiple packages with the same installed package id
 (shadowing) then the one which comes first in `GHC_PACKAGE_PATH` or which
 comes last on command line (according to `-package-db` flags) will be
 used.

 = The Problem =

 The build tool `Stack` implements stacked package databases. It uses a
 base package database and then stacks another package database on top of
 it to customise it further without modifying. The behavior is such that
 the package db on top of the stack completely overrides the ones below.
 That means you choose a package from top of the stack even if the version
 is older.

 Stack implements this by passing explicit package-ids of the packages to
 GHC. This scheme works well for cabal projects where we know ALL the
 packages used by the project in advance. But it does not work for scripts
 run using runghc. In that case we do not know the packages required by the
 script in advance and therefore cannot pass the package-ids to GHC. That
 means we cannot make GHC use the packages in the right way. GHC will
 choose the latest version even though we want it to choose a possibly
 older version from the top of the db stack.

 = Proposed Solution =

 Implement a new CLI option, something like `--stacked-pkg-dbs`. If this
 option is used GHC will use `GHC_PACKAGE_PATH` or the `-package-db`
 options to specify a stack of dbs.  The first db in the path or the last
 CLI option will be considered the top of the stack.

 The default behavior of GHC is to union all databases whereas this feature
 is about vertically stacking them. The following stacking rules will
 apply:

 * GHC will search a package from top to bottom and stop at the first db in
 which the package exists.
 * If GHC is not looking for a specific version then it will stop at the
 first db in which ANY version is found.
 * It will ignore the hidden packages when searching, but `-package <pkg>`
 will have the effect of unhiding all versions of `<pkg>`.
 * If there are multiple versions available in the same db then usual rules
 of latest non-broken package will be used to select one.
 * If the candidate package(s) found is broken it will not search further.
 When a broken package is needed in compilation it will report an error. It
 will not report errors in general when a broken package is encountered.

 This will allow us to modify an immutable package db by stacking another
 db on top. Implementing this as a separate option will keep the existing
 behavior so as to remain backward compatible.

 This has been discussed in a stack issue on github
 [https://github.com/commercialhaskell/stack/issues/1957 here].

--

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/12518#comment:5>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler