Request for feedback on spec/proposal for distributing package collections via hackage

[Duncan Coutts]

On Tue, 2015-07-14 at 12:02 -0700, Edward Z. Yang wrote:
> Hello Duncan,
> 
> In my eyes, this proposal looks like some sort of generalization of
> Stackage; and one further use case is "special purpose" collection.  My
> big question: how composable are these collections really?  I can't put
> two collections with conflicting versions together (or can I? Do I
> union?);

You're right that some use cases of collections only make sense if
clients can reasonably flexibly combine collections, e.g. set ops like
union, intersection and inversion or difference.

I think in principle taking collection unions or intersections makes
sense. For example, in stack, as I understand it, you can add a local
version of something that is also in the collection. This is of course a
union. So extending "narrow" collections by unioning them with extra
stuff makes sense. And suppose we had other "wide" collections that
didn't nail things down to just one version, then taking an intersection
with some other wide or narrow collection makes sense.

So in principle, these set-like operations make sense. The code we've
got in-progress for cabal-install allows exactly that. But none of that
is essential to make use of the general purpose collections like
stackage.

> and is there any point to having a collection without versions
> in it?

To be clear, every collection instance has a version. It's just that for
some of them -- live or rolling collections -- there is no particular
meaning to the version. So this thing about not specifying a version is
completely client side and doesn't affect the spec at all. It's just
worth pointing out as a use case.

As an example, we might have collections that are automagically defined
and updated based on some property of the packages. Those are unlikely
to have a meaningful version number.

> (If Cabal syntax is extended to support depending on collections
> as well as packages, yes?)

I don't think you're confused here, but just to clarify: packages do not
talk about collections. Tools like cabal/stack can be configured to use
collections.

And yes, where currently to configure cabal-install to use a collection
you have to explicitly list every member as a constraint in a
cabal.config file (see the cabal.config files distributed from the
stackage website), the extension in cabal-install is to support these
collections as a first class thing, by name-version or just name (and
with an expression language to combine them).

> The classic use-case for package collections is deployment settings, ala
> Stack, or even Cargo lockfiles / Bundler Gemfile.lock (versioned
> collections). In all these use-cases package collections are treated as
> non-compositional things.  http://doc.crates.io/guide.html
> http://bundler.io/v1.7/rationale.html#checking-your-code-into-version-control
> Libraries (compositional) do NOT publish lockfiles: only executables
> (non-compositional) DO.
> 
> Re the file format, it seems fine; suitable for the lockfile use-case
> and the Stackage use-case.  Less sure about the unioning semantics.

Right, collections and frozen settings are similar, and for the latter
composition doesn't make a lot of sense. We have the latter already of
course, in the form of "cabal freeze" cabal.config files, and similarly
for stack with .yml config files (which can be based off of pre-defined
collections). It's likely that cabal freeze will switch to use this
collection notation as it's somewhat more intentional (than the big sets
of raw constraints).

Duncan