planning for ghc-6.10.1 and hackage

[Duncan Coutts]

Hi,

We're getting pretty close to a final ghc-6.10.1 release. We would like
of course for the transition this time to be less painful than last
time. We all got a lot of flack last time for having no plan in place
and making everyone change all their .cabal files etc.

This time we can do a lot better. We have already decided to ship two
versions of base, version 3 and version 4. Version 4 is the even more
stripped down one, and with changes to exception handling etc. The
version 3.1 of base depends on base 4 and re-exports most of it with a
few differences so that it provides essentially the same api as base 3.0
did (which was included in ghc-6.8.x).

However we are not all the way there yet. If we released ghc-6.10.1
today everything would still break. By everything I mean everyone's
private projects and all the packages on hackage. The only packages that
would build would be trivial ones, or the ones that come with ghc.

There are two primary causes of this breakage:

      * cabal-installs dependency resolver does not work with base 3 & 4
      * Cabal will still build packages using base-4 rather than 3
        because most packages say "build-depends: base >= 2" rather than
        "build-depends: base ==3.*"

The current cabal-install dependency resolver is based around the idea
of picking at most one version of each package. This is because
typically using two versions of any package is a mistake. For base we
now have the situation that version 3 depends on version 4. So the
current resolver will find a conflict as soon as anything needs base 3.

The other problem is that packages currently typically specify an
optimistic upwardly open range rather than a pessimistic closed range.
Cabal uses the heuristic of picking the highest version of each package
that satisfies the version constraints.

I propose two solutions:

      * Fix the dependency resolver
      * Add support in Cabal and Hackage for suggested version
        constraints

The second is considerably easier than the first. The second involves
adjusting the dependency resolver to accept a per-package version
constraint, like "base < 4", "QuickCheck < 2" or "parsec < 3". It would
be a soft constraint that is used only when there is still a free choice
between multiple package versions. This should help the situation with
upward open version ranges like "build-depends: base". I have done this
first part but not tested it yet. Then we would add this list of
suggestions as an extra file in the 00-index.tar.gz file on hackage and
make cabal-install use it. Hopefully doing this would allow us to not
have to change every .cabal file on hackage, at least not immediately.
In tandem we should encourage (using automatic warnings) people to use
closed version ranges.

The first problem is quite hard. I could really do with some help on it.
The current dep resolver is a very simple constraint solving algorithm.
It's only as clever as was needed to make things work in most cases. It
is not a full general solver. It it based on being able to collect
version constraints per-package and end up with exactly one version of a
package. The obvious adaptation is to say that we can have multiple
slots for a single package, one for base 3 and one for base 4. However
it is not clear when we would decide to use two slots. Even if we just
hack it and hard code there being two for base, we do not know which
slot to put constraints into, probably both.

My concern here obviously is that if we release a ghc-6.10.1 and tell
everyone to start using it immediately for everything then we'll have
chaos and take yet more flack for breaking everything. It would also be
a great shame since Simon and Ian have already put a not-insignificant
amount of effort into the underlying mechanisms to allow us to provide a
compatibility base-3 package.

Remember we do have the ability to set up an extra hackage server with
everything and test how much builds or breaks with ghc-6.10. I'd like
the time to do a test run with that before the release so that we can
tell people what the expected level of breakage is and how to fix it.
I'm not doing that immediately because I already know nothing will work
atm without the two above issues being fixed.

So I'll aim to work on both issues this week and report back on how
things are going.

Duncan