QRE: Cabal and simultaneous installations of the same package

Tue May 12 19:58:13 UTC 2015

Vishal

Just to reiterate:
I think it would be incredibly helpful to have a wiki page in which the proposed design is fleshed out in detail.  The above is a start, but clearly I’m confused about many things.  Lacking a concrete, well-specified design we risk repeating the same conversations.  (I’m sure I have repeated above stuff that is well worn territory.)
I urge you strongly to begin your project by writing out your proposed design, and seeking feedback from the community.

Simon

From: Vishal Agrawal [mailto:vishal4556 at gmail.com]
Sent: 12 May 2015 18:54
To: Simon Peyton Jones
Cc: Boespflug, Mathieu; Gershom B; Mikhail Glushenkov; Duncan Coutts (duncan at well-typed.com); Ryan Trinkle; haskell-infrastructure at community.galois.com; cabal-devel at haskell.org; ghc-devs at haskell.org; Haskell Libraries
Subject: Re: QRE: Cabal and simultaneous installations of the same package

Yes, consistency means allowing only a single instance of a package in a environment.
A environment is very cheap to create than sandbox. Like every time a package is imported in ghci, a new environment is created which includes all the earlier package and new package.
Multiple environment can give sandbox for free.
Single database will have many benefits, although I don't think it is requirement. I plan to move to single database somewhat in middle.

On Tue, May 12, 2015 at 7:09 PM, Simon Peyton Jones <simonpj at microsoft.com<mailto:simonpj at microsoft.com>> wrote:
Enforcing consistency does have implications for the user interface. There will be situations where one wants to install a new package, but it is impossible to add it to the current environment while keeping all of the existing packages.

Let’s see if I understand this correctly.  I believe that Duncan/you envisage the following scenario:

•        Package database: you can install any package, compiled against any dependencies, without breaking anything.  So the package database can contain multiple instances of P-2.3, compiled against different dependencies.

•        Environment.   The “environment” maps a package name, such as “P-2.3” to a particular package instance in the database.  When you say “ghc –c Foo.hs –package P”, the “-package 2.3” consults the environment to determine which of the many P-2.3 instances you might mean.

Mind you, GHC’s new –package flag takes a package key (which identifies a unique package instance in the database).   So I’m hazy about exactly when the environment is used.

•        You can in principle have many environments (akin to sandboxes) but only one database.  There is a UI issue about how you define and maintain environments.

•        Duncan argues that, for predictability, every environment should be consistent.

o   I’m not sure exactly what “consistent” means.  Can I have P-2.3 and P-2.4 in the same environment, or only one P (for a given package name P)?

o   Intuitively, though, “consistent” means that all packages in the environment are compatible; they won’t give rise to the confusing “P-2.3:M.T does not match P-2.4:M.T” errors.

•        The environment is irrelevant when cabal is installing a package,.  Cabal just computes a set of consistent dependencies, installs any missing dependencies, and installs the package.  So the environment only matters for explicit user –package flags.
I think it would be incredibly helpful to have a wiki page in which the proposed design is fleshed out in detail.  The above is a start, but clearly I’m confused about many things.  Lacking a concrete, well-specified design we risk repeating the same conversations.  (I’m sure I have repeated above stuff that is well worn territory.)
Thanks!
Simon

From: Vishal Agrawal [mailto:vishal4556 at gmail.com<mailto:vishal4556 at gmail.com>]
Sent: 11 May 2015 23:02
To: Simon Peyton Jones
Cc: Boespflug, Mathieu; Gershom B; Mikhail Glushenkov; Duncan Coutts (duncan at well-typed.com<mailto:duncan at well-typed.com>); Ryan Trinkle; haskell-infrastructure at community.galois.com<mailto:haskell-infrastructure at community.galois.com>; cabal-devel at haskell.org<mailto:cabal-devel at haskell.org>; ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>; Haskell Libraries

Subject: Re: QRE: Cabal and simultaneous installations of the same package

Thanks. I will start working on it soon. Currently I am having a great mentorship by Ryan Trinkle. We have talked a few times, mostly about the plan of the project.

The main problem that can arise by the project is by enforcing consistency. It is already well documented by Duncan Coutts in a blog post from which I took most of the ideas from. http://www.well-typed.com/blog/2015/01/how-we-might-abolish-cabal-hell-part-2/

> Enforcing consistency does have implications for the user interface. There will be situations where one wants to install a new package, but it is impossible to add it to the current environment while keeping all of the existing packages. For example, suppose we have two different web stacks that have many packages in common but that require different versions of some common package. In that case we could not have a consistent environment that contains both. Thus the user interface will have to do something when the user asks to add the second web stack to an environment that already contains the first. The user interface could minimise the problem by encouraging a style of use where most environments are quite small, but it cannot be avoided in general.
> While what I am suggesting for consistency is relatively strong, we cannot get away without enforcing some restrictions on the environment. For example if our environment did contain two instances of the same version of a package then which one would we get when we launch GHCi? So my view is that given that we cannot avoid the user interface issues with environment consistency, it is better to go for the stronger and more useful form.
I am little unsure about what other community members think about it, but there is an acceptance from Duncan Coutts. If consistency is not enforced, errors like ‘bytestring is not bytestring’ can increase many times after implementing persistent package store as there will be a larger pool of packages. If enforced, there might be situation where developer will not be able load two package at the same time which can earlier work at the same time. My personal preference is to enforce consistency.

On 11-May-2015, at 9:47 pm, Simon Peyton Jones <simonpj at microsoft.com<mailto:simonpj at microsoft.com>> wrote:

Exactly!  It’s so close!  GHC has all the necessary infrastructure.  All we need is for Cabal to recognise that it’s possible to install two instances of A-2.0.  My fingers can almost touch it.  Go Vishal!

Simon

From: Boespflug, Mathieu [mailto:m at tweag.io]
Sent: 11 May 2015 16:42
To: Simon Peyton Jones; Gershom B; Mikhail Glushenkov; Duncan Coutts (duncan at well-typed.com<mailto:duncan at well-typed.com>); Ryan Trinkle; haskell-infrastructure at community.galois.com<mailto:haskell-infrastructure at community.galois.com>; cabal-devel at haskell.org<mailto:cabal-devel at haskell.org>; ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>; Vishal Agrawal; Haskell Libraries
Subject: Re: QRE: Cabal and simultaneous installations of the same package

As I think Simon was saying earlier, think of this feature as allowing strictly more installation plans to succeed, while still keeping to the exact same model that we use today of just one instance of a lib in any given binary.

Currently, if you install B-0.1 and then install A-2.0 that has a constraint B>=0.1, then you can't build an app that depends on both A and B-0.2. That's counter intuitive because had you started from an empty sandbox, then you would be able to build the app!

The reason is that currently you can only have a single instance of A-2.0 installed. The proposal is *not* to allow building an app against an A-2.0 built against B-0.1 and against B-0.2 simultaneously. It's to allow multiple instances of A-2.0 in the same package database, and teach Cabal to handle that, so that an app can ask for an A-2.0 that is built against the right version of B, no matter what, and link that in.

In your example, an app wouldn't get both C 1.0 and 2.0. It would get whichever one of those fits the constraints of both A and B, or the build will fail if no such C exists.

Since only one instance of a library ever makes it into a binary, as is the case currently, no particular problem arises with linking in external dependencies such as C code.

On 11 May 2015 at 14:52, Daniel Trstenjak <daniel.trstenjak at gmail.com<mailto:daniel.trstenjak at gmail.com>> wrote:

Hi Simon,

On Mon, May 11, 2015 at 11:27:58AM +0000, Simon Peyton Jones wrote:
> Well, no one is actually suggesting that!

But you're just getting it automatically if you're depending e.g.
on the libraries A and B, and A depends on C 1.0 and B depends on C 2.0.

Currently you can't build this dependency graph, right? And with
this proposed feature you will be getting C 1.0 and 2.0 into your binary.

Perhaps a even more nasty case is, if you're using a haskell library
which wraps a stateful c library, and now you're having multiple
versions of the same haskell library operating on the same c library state.

Greetings,
Daniel
_______________________________________________
ghc-devs mailing list
ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20150512/6317ac72/attachment-0001.html>