[Haskell-cafe] A new cabal odissey: cabal-1.8 breaking its own neck by updating its dependencies

[Duncan Coutts]

On 13 September 2010 20:54, Paolo Giarrusso <p.giarrusso at gmail.com> wrote:
> On Sun, Sep 12, 2010 at 20:46, Tillmann Rendel
> <rendel at mathematik.uni-marburg.de> wrote:
>> Paolo Giarrusso wrote:
>>> in a tracker entry you linked to,
>>> http://hackage.haskell.org/trac/hackage/ticket/704, duncan argues that
>>> "we also want to be able to do things like linking multiple versions
>>> of a Haskell package into a single application".
> [snip]
>> Even with the technical ability to link all of foo, bar, pair-1 and pair-2
>> together, I don't see how this program could be reasonably compiled.
>> Therefore, I think that the notion of consistent install plans is relevant
>> semantically, not just to work around some deficiency in the linking system.
>
> Your case is valid, but OTOH there other cases to support: if I link
> together two programs which use _internally_ different versions of
> regex packages, cabal should support that - and here I guess we agree.

Paolo,

If I've understood correctly, in this series of emails you're pointing
out two problems:

1. upgrading packages can break dependencies (and Cabal does not do a
lot to prevent/avoid this)

2. cabal ought to allow using multiple versions of a single package in
more circumstances than it does now


Both of these issues are known to the Cabal hackers (i.e. me and a few
other people). I'll share my views on the problem and the solution.

1. This is certainly a problem. The current situation is not awful but
it is a bit annoying sometimes. We do now accurately track when
packages get broken by upgrading dependencies so it should not be
possible to get segfaults by linking incompatible ABIs.

My preferred solution is to follow the example of Nix and use a
persistent package store. Then installing new packages (which includes
what people think of as upgrading) become non-destructive operations:
no existing packages would be broken by an upgrade. It would be
necessary to allow installing multiple instances of the same version
of a package.

If we do not allow multiple instances of a package then breaking
things during an upgrade will always remain a possibility. We could
work harder to avoid breaking things, or to try rebuilding things that
would become broken but it could never be a 100% solution.


2. This is a problem of information and optimisitic or pesimistic
assumptions. Technically there is no problem with typechecking or
linking in the presense of multiple versions of a package. If we have
a type Foo from package foo-1.0 then that is a different type to Foo
from package foo-1.1. GHC knows this.

So if for example a package uses regex or QC privately then other
parts of the same program (e.g. different libs) can also use different
versions of the same packages. There are other examples of course
where types from some common package get used in interfaces (e.g.
ByteString or Text). In these cases it is essential that the same
version of the package be used on both sides of the interface
otherwise we will get a type error because text-0.7:Data.Text.Text
does not unify with text-0.8:Data.Text.Text.

The problem for the package manager (i.e. cabal) is knowing which of
the two above scenarios apply for each dependency and thus whether
multiple versions of that dependency should be allowed or not.
Currently cabal does not have any information whatsoever to make that
distinction so we have to make the conservative assumption. If for
example we knew that particular dependencies were "private"
dependencies then we would have enough information to do a better job
in very many of the common examples.

My preference here is for adding a new field, build-depends-private
(or some such similar name) and to encourage packages to distinguish
between their public/visible dependencies and their private/invisible
deps.

Duncan