Hackage is flooded with old package versions reuploads

[Herbert Valerio Riedel]

On 2015-01-18 at 18:56:42 +0100, kyra wrote:
> It looks old (and even ancient) versions of many packages gets
> uploaded to hackage over and over again in ever increasing
> amounts. The username of uploader for vast majority of these uploads
> is HerbertValerioRiedel.

That would be me...

Just to clarify: I those aren't re-uploads of existing package versions,
but rather meta-data revisions. The actual source-code is *not* touched,
(and Hackage is very restricting in what's allowed to be changed in a
.cabal file)

This is also mentioned on http://hackage.haskell.org/packages/trustees/

> While this is harmless I wonder what idea stands behind this?

The idea, as others in this thread have already pointed out, is to add
missing lower/upper package bounds to packages so that the Cabal
install-plan solver can fulfill its task.

Over the last week, I've been analyzing the current situation (motivated
by the upcoming GHC 7.10 release which trips up quite a few packages
with missing bounds on base/deepseq/time), and trying to come up with
some tooling[1] to help the otherwise IMO rather unmanageable job of a
Hackage trustee, as well as surgically fixing up the meta-data where I
had empirical proof that the .cabal meta-data was incorrect (and thus
leading the solver to select a broken install-plan).

Here's a very simple buildreport for illustration:

 https://ghc.haskell.org/~hvr/buildreports/pretty.html

The purpose of my meta-data edits were/are (in order of decreasing
priority):

 1. First and foremost, no currently working/valid install-plan was lost.

 2. A simple constraint-less `cabal install pretty` ought to result
    *either* in a valid install-plan, *or* cabal failing to find any
    install-plan.

 3. An major-version constraint `cabal install 'pretty == a.b.*'` ought
    to result *either* in a valid install-plan, *or* cabal failing to find
    any install-plan.

 4. A package shall not cause indirect *build*-failures of other
    packages directly or indirectly depending on this package[3]

The build-report linked helps directly with 2. and 3. as it shows that
`pretty` failed to `cabal install pretty` (as well as `cabal install
'pretty == 1.1.*'`) for GHC 7.0 and GHC 7.2.  Case 4. could be detected
by running the buildtool on many Hackage packages depending on `pretty`,
and look for failures to build `pretty` during the `cabal install --dep`
step (which would result in a "FAIL (deps)" cells).


I've encountered a couple of situations, where fixing the meta-data of
an important package low in the dependency graph fixed a couple of other
packages depending on that package. However, this also means that the
more popular a package is, the higher its responsibility is to make sure
its build-depend meta-data is correct, as such packages have the
potential to instantly break large branches of Hackage:

One example for that was the release of deepseq-1.4, which changes the
default-implementation of `rnf` which a few packages relied on, but the
rather important `text` package was among of those. This affected most
of the released ~50 `text` package versions, and since `deepseq` is
compatible with all GHC 7.* versions, as soon an install-plan was
selected which included `deepseq-1.4` with a non-latest version of
`text` the build would fail. And since it's not uncommon to restrict the
upper bound of `text`[2], all those packages depending on a non-latest
`text` would have broken install-plans (unless `deepseq < 1.4` was
selected).

I hope this answers your question.

Cheers,
  hvr


 [1]: https://github.com/hvr/hackage-matrix-builder

      (WIP example reports can be seen in

       https://ghc.haskell.org/~hvr/buildreports/0INDEX.html)


 [2]: http://packdeps.haskellers.com/reverse/text

 [3]: This can happen due to additional package-version constraints
      causing other (but broken) configurations to be selected by the
      solver not detected in case 2. and 3.