planning for ghc-6.10.1 and hackage [or: combining packages to
yield new type correct programs]
dons at galois.com
Thu Oct 2 03:58:09 EDT 2008
> We're getting pretty close to a final ghc-6.10.1 release. We would like
> of course for the transition this time to be less painful than last
> time. We all got a lot of flack last time for having no plan in place
> and making everyone change all their .cabal files etc.
> This time we can do a lot better. We have already decided to ship two
> versions of base, version 3 and version 4. Version 4 is the even more
> stripped down one, and with changes to exception handling etc. The
> version 3.1 of base depends on base 4 and re-exports most of it with a
> few differences so that it provides essentially the same api as base 3.0
> did (which was included in ghc-6.8.x).
> However we are not all the way there yet. If we released ghc-6.10.1
> today everything would still break. By everything I mean everyone's
> private projects and all the packages on hackage. The only packages that
> would build would be trivial ones, or the ones that come with ghc.
Here are my notes on why this is so hard, and so scary, and what we can do.
So Duncan and I spent about 6 hours tonight working out how to make the
cabal-install constraint solver, and the Cabal configure invariants,
work when programs attempt to use base-3 and base-4 on the same system.
Here's a summary of why this is non-trivial,
* We're trying to compose packages on the users machine to yield new
type correct programs.
* We're using cabal dependencies to decide when it is safe to do this.
Hopefully we don't rule in any type incorrect combinations, nor rule
out to many type correct combinations.
* This is scary - in fact, we think the package system admits type
incorrect programs (based on Typeable, or initialised global state),
as it is similar to the runtime linking problem for modules.
* We use constraint solving determine when composition is safe, by looking at
"package >= 3 && < 4"
style constraints. That is, we try to guess when the composition
would yield a type correct program.
* Again, we're using constraint solving on this language to
determine when composition of Haskell module sets (aka packages) would
yield type correct Haskell programs
All without attempting to do type checking of the interfaces between
packages -- the very thing that says whether this is sound!
* This previously relied on a strong invariant:
+ There was only to be one package-version assignment in a program
+ People didn't change APIs too much, and tried to follow the
versioning spec, and wrote .cabal deps that followed the spec.
* But now we allow multiple package-version assignments, as long as one
depends on the other, and doesn't redefine types.
(You'll notice by now we're in deep scary-land, trying, after compile
time, to compose type correct modules into sets of modules (packages),
and then composing those sets with each other to yield type correct
programs, some of which redefine parts of the module hierarchy, and
doing all this without implementing a type checking story for when to
combine these sets safely).
* So, the solver for cabal-install has to be updated to allow the same
package to have multiple, conflicting versions, as long as version X
depends on version Y, and then not reject programs that produce this
* This is non trivial, but think this refactoring is possible, but it is
hard. ultimately we're still making optimistic assumptions about
when module sets can be combined to produce type correct programs,
and conservative assumptions, at the same time.
What we need is a semantics for packages, that in turn uses a
semantics for modules, that explains interfaces in terms of types.
Packages of functors, containing modules, which we can truly compose
to yield type correct programs.
* But we think the cabal-install solver will be refactorable to let a
good number of programs work, but it'll take a few days of constraint solver hacking.
* The end result is that cabal-install should be able to find automated
install plans for packages that ask for base-3, even when base-4 is on
the system as well, and it uses pieces of base-3 libraries and base-4
libraries. Some more programs will work than if we didn't ship base-3.
* The two other ways of combining packages into Haskell programs,
+ ghc --make
+ runhaskell Setup.hs configure
will always pick base-4 now, so you won't be able to build programs
against the base-3 without manually overriding. This means things that
need the old exception API, for example, or syb, will break without
manual hiding (--package base-22.214.171.124).
* We're missing hard stats on what number of things break under which
More information about the Glasgow-haskell-users