[Haskell-cafe] Platform Versioning Policy: upper bounds are not our friends

[Michael Sloan]

Upper bounds are a bit of a catch-22 when it comes to library authors evolving
their APIs:

1) If library clients aren't encouraged to specify which version of the
   exported API they target, then changing APIs can lead to opaque compile
   errors (without any information about which API is intended).  This could
   lead the client to need to search for the appropriate version of the
   library.

   If library clients are encouraged to specify which versions of the
exported API they target,
  then changing the API breaks all of the clients.

There are a couple of hacky ways to do #1 without having the errors be so opaque

1) Start a tradition of commenting the cabal dependencies with the version of
   the package that worked for the author.

2) Build in support for these "known good" versions into cabal, perhaps
   generated on "sdist"release, or with a particular build flag.  (they don't
   need to be stored in the .cabal file)

3) Attempt to post-process GHC error messages to guess when the issue might be
   caused by the package version being different according to the PVP.  This
   could work alright for scoping but wouldn't work well for type errors
   (which matter more)

I like the idea of doing automated upper-bound-determination!  That would be
very convenient. It's a bit tricky though - should tests be included?

I think the most ideal solution is to attack the core problem: things break
when dependencies change their interface.  This is pretty expected in the
development world at large, but I think that Haskell can do better.

The main idea is that package developers should be free to update their API as
they realize that new names or interfaces are better.  Currently there's the
problem that the APIs that actually get used subsequently stagnate due to fear
of breakage.

The solution that I think makes the most sense is to start exporting modules
which express the old interface in terms of the new.  What's interesting about
this is that most non-semantic changes to things other than ADTs and
typeclasses can already be expressed in plain Haskell code.

This idea, among other things, inspired the "instance templates" proposal,
which is somewhat related to superclass default instances.  With this language
extension, it would be possible to express compatibility layers for instance
definitions, something that is impossible with Superclass Default Instances.

https://github.com/mgsloan/instance-templates

I've also got a start on a utility for extracting API signatures from
packages.  Currently it just pretty prints the API in a fashion that attempts
to be amenable to textual diffing:

https://github.com/mgsloan/api-compat/
https://github.com/mgsloan/api-compat/blob/master/examples/diagrams-core.api.diff

The intent is to make the tool interactive, giving the user a chance to let
the tool know which exports / modules have been renamed.  After the user
provides this information, it should be possible to generate almost all of the
compatibility code.

In order to make it convenient to use these compatibility modules, we'd want
to have some cabal- invoked code generation that would generate proxy modules
that re-export the appropriate version. This could all happen in a separate
hs-source-dir.

The next step in this toolchain is something that's very hard to do nicely,
because it can change code layout: automated rewriting of user code to target
the new version (this is equivalent to "inlining" the compatibility module
definitions).  However, even a tool that would take you to all of the
places that
need changing would be invaluable.

Wouldn't it be excellent, if the Haskell eco-system managed something that no
other language affords?:

* Automatic refactoring to target new API versions

* Expression of these refactorings in the language itself

* Rigorous, structured documentation of inter-version changes.  This'd also
  provide a nice place to put haddocks with further change information.

-Michael

On Wed, Aug 15, 2012 at 2:34 PM, Carter Schonwald
<carter.schonwald at gmail.com> wrote:
> As someone who recurrently is nudging a large number of maintainers every
> major ghc release to bump their bounds, I favor the no upper bounds
> approach! :)
>
> plus the whole improving ecosystem of build bot tools which play nice with
> cabal et al that are cropping up mean that "in principal" we could debug
> missing upper bounds via sort of temporal bisecting over the "event stream"
> of maximum available versions at a given time to sort that. (but that piece
> isn't that important)
>
> more pragmatically, cabal when used with hackage doesn't let you override
> version constraints, it just lets you add additional constraints. This makes
> sense if we assume that the library author is saying "things will definitely
> break if you violate them", but in practice upper bounds are made up
> guesstimation.
>
> YES, its presumably semantic versioning doesn't create a problem, but with
> the hackage eco system, when dealing with intelligently engineering libs
> that are regularly maintained, version upper bounds create more problems
> than than solve.
>
> just my two cents. (yes yes yes, please drop upper bounds!)
>
> cheers
> -Carter
>
>
> On Wed, Aug 15, 2012 at 5:04 PM, Michael Blume <blume.mike at gmail.com> wrote:
>>
>> > it's usual for the existing upper bounds to refer to versions that don't
>> > exist at the time of writing (and hence can't be known to be stable).
>>
>> Well, known to be stable given semantic versioning, then.
>>
>> http://semver.org/
>>
>> On Wed, Aug 15, 2012 at 1:55 PM, Bryan O'Sullivan <bos at serpentine.com>
>> wrote:
>> > On Wed, Aug 15, 2012 at 1:50 PM, David Thomas <davidleothomas at gmail.com>
>> > wrote:
>> >>
>> >> Would it make sense to have a known-to-be-stable-though soft upper
>> >> bound
>> >> added proactively, and a known-to-break-above hard bound added
>> >> reactively,
>> >> so people can loosen gracefully as appropriate?
>> >
>> > I don't think so. It adds complexity, but more importantly it's usual
>> > for
>> > the existing upper bounds to refer to versions that don't exist at the
>> > time
>> > of writing (and hence can't be known to be stable).
>> > _______________________________________________
>> > Haskell-Cafe mailing list
>> > Haskell-Cafe at haskell.org
>> > http://www.haskell.org/mailman/listinfo/haskell-cafe
>> >
>>
>> _______________________________________________
>> Haskell-Cafe mailing list
>> Haskell-Cafe at haskell.org
>> http://www.haskell.org/mailman/listinfo/haskell-cafe
>
>
>
> _______________________________________________
> Haskell-Cafe mailing list
> Haskell-Cafe at haskell.org
> http://www.haskell.org/mailman/listinfo/haskell-cafe
>