[Haskell-cafe] A new cabal odissey: cabal-1.8 breaking its ownneck by updating its dependencies

[Claus Reinke]

On the topic of cabal odisseys:

I think it would help to document (prominently) what Cabal 
fundamentally doesn't (try to) do, to avoid optimistic 
expectations (and hence the surprises when Cabal doesn't
meet those expectations), and to point out the design choices 
behind many bug tickets (even if the choices are temporary
and driven by limited manpower). Such as

- cabal doesn't keep track of what it installs, hence

    - no uninstall
    - no application tracking
    - library tracking and information about installed
        library configurations only via ghc-pkg
    ..

- cabal's view of package interfaces is limited to explicitly
    provided package names and version numbers, hence

    - no guarantee that interface changes are reflected in
        version number differences
    - no detailed view of interfaces (types, functions, ..)
    - no reflection of build/configure options in versions
    - no reflection of dependency versions/configurations
        in dependent versions
    - no knowledge of whether dependencies get exposed
        in dependent package interfaces
    ..

This is just to demonstrate the kind of information I'd like
to see - Duncan&co know the real list, though they don't
seem to have it spelled out anywhere? So users see that
it works to say "cabal install" and build up their own often
optimistic picture of what (current) Cabal is supposed to
be able to do. It might even be useful to have a category
of "core-tickets" or such on the trac, to identify these as
work-package opportunities for hackathons, GSoC and 
the like, positively affecting many tickets at once.

On to the specific issue at hand:

> 2. cabal ought to allow using multiple versions of a single package in
> more circumstances than it does now
>..
> 2. This is a problem of information and optimisitic or pesimistic
> assumptions. Technically there is no problem with typechecking or
> linking in the presense of multiple versions of a package. If we have
> a type Foo from package foo-1.0 then that is a different type to Foo
> from package foo-1.1. GHC knows this.
> 
> So if for example a package uses regex or QC privately then other
> parts of the same program (e.g. different libs) can also use different
> versions of the same packages. There are other examples of course
> where types from some common package get used in interfaces (e.g.
> ByteString or Text). In these cases it is essential that the same
> version of the package be used on both sides of the interface
> otherwise we will get a type error because text-0.7:Data.Text.Text
> does not unify with text-0.8:Data.Text.Text.
> 
> The problem for the package manager (i.e. cabal) is knowing which of
> the two above scenarios apply for each dependency and thus whether
> multiple versions of that dependency should be allowed or not.
> Currently cabal does not have any information whatsoever to make that
> distinction so we have to make the conservative assumption. If for
> example we knew that particular dependencies were "private"
> dependencies then we would have enough information to do a better job
> in very many of the common examples.
> 
> My preference here is for adding a new field, build-depends-private
> (or some such similar name) and to encourage packages to distinguish
> between their public/visible dependencies and their private/invisible
> deps.

This private/public distinction should be inferred, or at least be 
checked, not just stated. I don't know how GHC computes its
ABI hashes - are they monolithic, or modular (so that the 
influence of dependencies, current package, current compiler,
current option settings, could be extracted)? 

Even for monolithic ABI hashes, it might be possible to compute 
the package ABI hash a second time, setting the versions of all 
dependencies declared to be private to 0.0.0 and seeing if that 
makes a difference (if it does, the supposedly private dependency 
leaks into the package ABI, right?).

And secondly, how about making it possible for cabal files to 
express sharing constraints for versions of dependencies?

To begin with, there is currently no way to distinguish packages 
with different flag settings or dependency versions. If I write a 
package extended-base, adding the all-important functions car 
and cdr to an otherwise unchanged Data.List, that package will 
work with just about any version of base (until car and cdr 
appear in base:-), but the resulting packages may not be 
compatible, in spite of identical version numbers.

If it was possible to refer to those package parameters (build 
options and dependency versions), one could then add constraints
specifying which package parameters ought to match for a build
configuration to be acceptable.

Let us annotate package identifiers with their dependencies
where the current lack of dependency and sharing annotations 
means "I don't care how this was built". Then

    build-depends:  a, regex

means "I need a and regex, but I don't care whether a also uses
some version of regex", while

    build-depends: a, regex
    sharing: a(regex)==regex

would mean "I need any version of a and regex, as long as a 
depends on same the version of regex I depend on" (choose
any syntax that works). And

    build-depends: a, b
    sharing: a(text)==b(text)

would mean "I need any version of a and b, as long as they
both depend on the same version of text".

I can already think of situations where one might want more
complex constraints (e.g. "I want any version of base and mtl,
but either both have to be before the Either-move, or both
have to be after the move" - this part could be expressed
already, but there is currently no way to enforce this 
transitively, for dependencies, so if I depend on c, which
depends on a new base, and on d, which depends on an 
old mtl, I'm out of luck, I think). Or: "never mix mtl and
its alternatives".

Of course, this scheme depends on whether it is possible
to reverse engineer the information about dependencies of
installed packages from ghc-pkg info or ABI hashes.. but
if the general idea is sound, perhaps that could be made
possible?

Just a suggestion,
Claus