Why upper bound version numbers?

[Herbert Valerio Riedel]

On 2016-06-07 at 02:58:34 +0200, Dominick Samperi wrote:
> I guess what you are saying is that this policy will prevent packages
> from installing with new versions of ghc until the maintainer has had
> a chance to test the package with the new version, and has updated the
> upper version limit. Thus, inserting those upper version limits is a
> kind of flag that indicates that the package has been "certified" for
> use with versions of base less than or equal to the upper limit.

That's one important aspect. I'm very distrustful of packages whose
maintainers declares that their packages have eternal future
compatibility (unless they have made this decision *very* carefully
based on which parts of the API they use).

In general, this runs into the fallacy that successful compilation would
be equivalent to (semantic) API compatibility, which is only half the
story. In some cases one may be lucky to get compile-time warnings
(which are often ignored anyway), or explicit run-time errors (which are
still undesirable), or even worse silent failures where the code behaves
subtly wrong or different than expected. Testsuites mitigate this to
some degree, but they too are an imperfect solution to this hard
problem.

So another aspect is that the PVP[1] provides an API contract which makes
upper bounds possible at all (for packages you don't control).

While the PVP doesn't give you a way to know for sure when compatibility
breaks, the policy gives you a least upper bound up to which your
package is guaranteed (under certain conditions) to remain
compatible. Without this contract, you'd have no choice but to
constraint your package to versions of dependencies (not under your
control) which you were able to empirically certify to be compatible
semantically, i.e. versions that were already published.

Unfortunately, GHC's `base` package has a *huge* API surface. So with
each GHC release we're usually forced to perform a major version bump to
satisfy the PVP, even if just a tiny part only very few packages use of
`base`'s API became backward in-compatible. This may be addressed by
reducing the API surface of `base` by moving infrequently used
GHC-internal-ish parts of the API out of base.

But there's also other changes which affect many more packages as
already mentioned. As already mentioned, the big AMP change was a major
breaking point.

Some packages like

  http://matrix.hackage.haskell.org/package/unordered-containers

tend to break every time a new GHC version comes out. Partly because they
happen to use the low-level parts of `base` API which tend to
change.

Ironically, in the case of `unordered-containers`, the maintainer decided
to start leaving off (or rather make ineffective) the upper bound on
`base` starting with 0.2.2.0, and this turned to be an error in
judgment, as each time a new GHC version came out, the very bound that
was left out would turned out to be necessary. So leaving off upper
bounds created actually more work and no benefit in the case of
`unordered-containers`.



 [1]: https://wiki.haskell.org/Package_versioning_policy