[Haskell-cafe] Hackage inconsistent? (pandoc.cabal in 1.13.1)

[Andreas Abel]

On 28.10.2014 01:38, Mateusz Kowalczyk wrote:
 > 4. I make changes to ap-1.0.0.0, Hackage generates ap-1.0.0.0-fix1
 > package with identical content but new info without any extra effort on
 > my side.

Basically you suggest that there should be a last digit in the version 
number that is only used for dependency fixes and is handled 
automatically by cabal and hackage.  We could as well say that Hackage 
generates

   ap-1.0.0.0.1
   ap-1.0.0.0.2  -- for the next dependency fix

but you cannot specify the 5th digit in cabal commands, so when you say

   cabal install ap-1.0.0.0

you get the latest, i.e. ap-1.0.0.0.2.

--Andreas

On 28.10.2014 01:38, Mateusz Kowalczyk wrote:
> On 10/27/2014 07:48 PM, Erik Hesselink wrote:
>> I was a bit hesitant about using this feature as well. We've had some
>> confusion when we used it [1][2][3]. The Nix guys also seem to be
>> having some trouble fitting it into their workflow [4]. But in [1] I
>> did a quick grep to see how many packages used the feature, and there
>> are already quite a lot. There is an 'x-revision' property added to
>> the cabal file in the index, so in theory I guess tools could learn to
>> deal with it.
>>
>> In general I think the biggest win for this feature is not even in
>> relaxing bounds, but in tightening them when it turns out they're too
>> loose. Previously cabal was always free to pick an old, unconstrained
>> version and would often do so, leading to type errors during builds.
>>
>> Erik
>>
>> [1] https://github.com/silkapp/rest/issues/57
>> [2] https://github.com/silkapp/aeson-utils/issues/1
>> [3] https://github.com/silkapp/json-schema/issues/12
>> [4] https://github.com/NixOS/cabal2nix/issues/84
>>
>> On Mon, Oct 27, 2014 at 8:30 PM, John MacFarlane <jgm at berkeley.edu> wrote:
>>> I must say, I did pause a bit before making this change to
>>> pandoc on Hackage, precisely because I realized that the
>>> cabal file in the package would not match the one on
>>> Hackage.
>>>
>>> But I went ahead, assuming that if the means were provided
>>> to make this kind of change, it should be okay.
>>>
>>> It is time-consuming for me to make a proper pandoc
>>> release.  I need to write up release notes, rebuild the
>>> website, generate and test the binary packages on various
>>> different platforms, upload to github releases and Hackage,
>>> send announcement emails, etc.  Even with automation this
>>> all takes work.  I'd rather not have to do this every time a
>>> version bound changes on a dependent package, so I like this
>>> way of making minor tweaks to version bounds; I hope I
>>> can be persuaded that it's not a bad idea.
>>>
>>> John
>>>
>
> TL;DR version of below: I think Hackage should generate a new package
> version for each info change rather than the change being ‘silent’
> making it pretty exclusive to tools which don't care about what they
> actually build and what hash it has (cabal). This brings us back to ‘all
> version are explicit’ from the ‘we have versions but there are some
> updates you need to pull in your tools too’.
>
> Long version:
>
> As Erik points out at his [4], this really messes things up for us in
> the nix world, and really anywhere that
>
> 1. Wants to grab from Hackage
> 2. Wants a reproducible build
>
> The common justification for this is that it doesn't change the tarball
> hash. Well then, this ‘update info after upload’ feature then
> automatically excludes every single system which uses this tarball. Why?
>
> 1. Maintainer uploads a package
> 2. Distros package it and note down the hash
> 3. Something happens and maintainer bumps the bounds on Hackage only
> 4. Distros still grab the very same package as they did before (tarball
> doesn't change, remember?). Inside is the cabal file with old
> dependencies, so the build fails if the distro updated other packages
> (almost certainly).
>
> So even though the hash doesn't change, builds suddenly start failing.
> If you're using cabal, you don't get a reproducible build at all because
> which versions it picks depends on whatever info is on Hackage at build
> time (well, last cabal update time) and not what's in the tarball. It is
> currently perfectly possible for two separate users to have ‘cabal
> update’ and know about exactly the same package versions available but
> end up with different version numbers used because one ‘cabal update’d
> before the info on Hackage was changed and the other did not.
>
> But John is right that sometimes making releases is tiresome. For
> version bumps it can be as easy as checking out the last tag, bumping
> deps and uploading, but it can be much more involved too. I admit that I
> used this myself, although only for the stuff that hope to release
> properly soon and I happen to maintain on NixOS too which AFAICT is the
> only distro which packages Haskell fast enough to notice breakage within
> few days (in huge part thanks to Peter Simons ;) )
>
> So clearly there is a problem here: only cabal users use the information
> from Hackage info bumps. Everyone else who looks into the tarball just
> fails to build. cabal users who happen to cabal update at unfortunate
> time also fail to build.
>
> I can only think of one ‘solution’ which fixes this and doesn't require
> considerable effort and co-ordination between maintainers and packagers.
> Here is how situation currently looks like
>
> 1. I upload awesomepackage (ap) to Hackage under ap-1.0.0.0
> 2. Time passes, I have commits on top
> 3. One of those packager type of folk who like to demand things come
> onto my issue tracker and say I should update my dependencies but I
> don't want to release right now
> 4. I go onto Hackage and use that new cool feature, changing the cabal
> file on ap-1.0.0.0.
> 5. Packager is unhappy because the problem is not fixed, ap-1.0.0.0
> gives them the same thing. The only thing they can do is monkeypatch
> deps on their side.
>
> Instead, I think at steps 4 and 5 the following should happen:
>
> 4. I make changes to ap-1.0.0.0, Hackage generates ap-1.0.0.0-fix1
> package with identical content but new info without any extra effort on
> my side.
> 5. Packager simply now points to ap-1.0.0.0-fix1 which has the new hash
> and builds great again. cabal does The Right Thing™ here and resolves
> ap-1.0.0.0 to ap-1.0.0.0-fix1 so everything keeps working there as it
> does now.
>
> It does seem a bit wasteful space-wise but with clever programming on
> Hackage side that could probably be avoided. There is no longer a
> problem with these ‘hidden’ changes happening. Everyone can point at the
> *exact* version they want used like they do now and they get what
> exactly what they asked for. A less wasteful way would be for the
> packaging distros to update their tools to look at the Hackage-only
> cabal files as well. Personally I think that seems like the wrong place
> to solve this though.
>
> Lastly, Hackage should announce with big fat red letters on the front
> page of the package that the source tarball contains different info than
> what's shown on Hackage. It's really not possible to tell as a user at
> the moment unless we specifically look for it.
>
> Sorry for the long-winded message, hopefully you heeded the TL;DR at the
> top and skipped this.
>


-- 
Andreas Abel  <><      Du bist der geliebte Mensch.

Department of Computer Science and Engineering
Chalmers and Gothenburg University, Sweden

andreas.abel at gu.se
http://www2.tcs.ifi.lmu.de/~abel/