Libraries in the repo

[Judah Jacobson]

On Wed, Aug 26, 2009 at 9:15 AM, Simon Marlow<marlowsd at gmail.com> wrote:
> Simon and I have been chatting about how we accommodate libraries in the GHC
> repository.  After previous discussion on this list, GHC has been gradually
> migrating towards having snapshots of libraries kept as tarballs in the repo
> (currently only "time" falls into this category), but I don't think we
> really evaluated the alternatives properly.  Here's an attempt to do that,
> and to my mind the outcome is different: we really want to stick to having
> all libraries as separate repositories.
>
> Background:
>  * Scope: libraries that are needed to build GHC itself (aka "boot
>   libraries")
>
>  * Boot libraries are of several kinds:
>   - INDEPENDENT: Independently maintained (e.g. time, haskeline)
>   - COUPLED: Tightly coupled to GHC, but used by others (base)
>   - SPECIFIC: Totally specific to GHC (e.g. template-haskell, DPH)
>
> Choices for how we deal with libraries in the GHC repository: (+) is a
> pro, (-) is a con.
>
>  (1) Check out the library from a separate repo, using the darcs-all
>      script.  The repo may either be a GHC-specific branch
>      [INDEPENDENT], or the master copy of the package
>      [SPECIFIC/COUPLED].
>
>  (2) Put a snapshot tarball of the library in libraries/tarballs,
>      but allow you to checkout the darcs repo instead.
>
>  (3) The GHC repo *itself* contains every library unpacked in the
>      tree.  You are allowed to check out the darcs repo instead.
>
>
> (3) works best for SPECIFIC libraries, whereas (1) works best for
> INDEPENDENT/COUPLED libraries.  If we want to treat all libraries the
> same, then the only real option is (1).

Agreed.  Also, it seems odd to have template-haskell be built-in yet
something so fundamental as base be a separate repo.

> Experience with Cabal and bytestring has shown that (1) can work for
> INDPENDENT libraries, but only if we're careful not to get too
> out-of-sync (as we did with bytestring).  In the case of Cabal, we never
> have local changes in our branch that aren't in Cabal HEAD, and that works
> well.
>
> Comments/thoughts?

I also would rather stay with (1).  Although using a DVCS allows
greater freedom for developers, it also creates the need for more
explicit rules of process.  So I propose codifying on the wiki that
for certain libraries, the local ghc repo

 - Never has patches which are not in the library's HEAD
 - Pulls patches sparingly, and usually only after a tagged release of
the library.

(the darcs-all script could help double-check that the former is being obeyed.)

We package admins would need to agree to be responsive to patch
submissions from GHC devels (or grant push access to them).

Thanks for your very helpful analysis,
-Judah