[Haskell-cafe] Hackage inconsistent? (pandoc.cabal in 1.13.1)

Tue Oct 28 21:22:26 UTC 2014

per se, thats what hackage DOES now. Though it is valid to want some sort
of +patchlevel suffix to be visible for distro engineering purposes or such.

Speaking as a hackage trustee, this .cabal editing facility has been a VERY
VERY good thing overall. A lot of problems have been prevented because of
it, and that its creating new problems that haven't existed before should
be considered a testament to its success! ;)

On Tue, Oct 28, 2014 at 8:17 AM, Andreas Abel <andreas.abel at ifi.lmu.de>
wrote:

> On 28.10.2014 01:38, Mateusz Kowalczyk wrote:
> > 4. I make changes to ap-1.0.0.0, Hackage generates ap-1.0.0.0-fix1
> > package with identical content but new info without any extra effort on
> > my side.
>
> Basically you suggest that there should be a last digit in the version
> number that is only used for dependency fixes and is handled automatically
> by cabal and hackage.  We could as well say that Hackage generates
>
>   ap-1.0.0.0.1
>   ap-1.0.0.0.2  -- for the next dependency fix
>
> but you cannot specify the 5th digit in cabal commands, so when you say
>
>   cabal install ap-1.0.0.0
>
> you get the latest, i.e. ap-1.0.0.0.2.
>
> --Andreas
>
>
> On 28.10.2014 01:38, Mateusz Kowalczyk wrote:
>
>> On 10/27/2014 07:48 PM, Erik Hesselink wrote:
>>
>>> I was a bit hesitant about using this feature as well. We've had some
>>> confusion when we used it [1][2][3]. The Nix guys also seem to be
>>> having some trouble fitting it into their workflow [4]. But in [1] I
>>> did a quick grep to see how many packages used the feature, and there
>>> are already quite a lot. There is an 'x-revision' property added to
>>> the cabal file in the index, so in theory I guess tools could learn to
>>> deal with it.
>>>
>>> In general I think the biggest win for this feature is not even in
>>> relaxing bounds, but in tightening them when it turns out they're too
>>> loose. Previously cabal was always free to pick an old, unconstrained
>>> version and would often do so, leading to type errors during builds.
>>>
>>> Erik
>>>
>>> [1] https://github.com/silkapp/rest/issues/57
>>> [2] https://github.com/silkapp/aeson-utils/issues/1
>>> [3] https://github.com/silkapp/json-schema/issues/12
>>> [4] https://github.com/NixOS/cabal2nix/issues/84
>>>
>>> On Mon, Oct 27, 2014 at 8:30 PM, John MacFarlane <jgm at berkeley.edu>
>>> wrote:
>>>
>>>> I must say, I did pause a bit before making this change to
>>>> pandoc on Hackage, precisely because I realized that the
>>>> cabal file in the package would not match the one on
>>>> Hackage.
>>>>
>>>> But I went ahead, assuming that if the means were provided
>>>> to make this kind of change, it should be okay.
>>>>
>>>> It is time-consuming for me to make a proper pandoc
>>>> release.  I need to write up release notes, rebuild the
>>>> website, generate and test the binary packages on various
>>>> different platforms, upload to github releases and Hackage,
>>>> send announcement emails, etc.  Even with automation this
>>>> all takes work.  I'd rather not have to do this every time a
>>>> version bound changes on a dependent package, so I like this
>>>> way of making minor tweaks to version bounds; I hope I
>>>> can be persuaded that it's not a bad idea.
>>>>
>>>> John
>>>>
>>>>
>> TL;DR version of below: I think Hackage should generate a new package
>> version for each info change rather than the change being ‘silent’
>> making it pretty exclusive to tools which don't care about what they
>> actually build and what hash it has (cabal). This brings us back to ‘all
>> version are explicit’ from the ‘we have versions but there are some
>> updates you need to pull in your tools too’.
>>
>> Long version:
>>
>> As Erik points out at his [4], this really messes things up for us in
>> the nix world, and really anywhere that
>>
>> 1. Wants to grab from Hackage
>> 2. Wants a reproducible build
>>
>> The common justification for this is that it doesn't change the tarball
>> hash. Well then, this ‘update info after upload’ feature then
>> automatically excludes every single system which uses this tarball. Why?
>>
>> 1. Maintainer uploads a package
>> 2. Distros package it and note down the hash
>> 3. Something happens and maintainer bumps the bounds on Hackage only
>> 4. Distros still grab the very same package as they did before (tarball
>> doesn't change, remember?). Inside is the cabal file with old
>> dependencies, so the build fails if the distro updated other packages
>> (almost certainly).
>>
>> So even though the hash doesn't change, builds suddenly start failing.
>> If you're using cabal, you don't get a reproducible build at all because
>> which versions it picks depends on whatever info is on Hackage at build
>> time (well, last cabal update time) and not what's in the tarball. It is
>> currently perfectly possible for two separate users to have ‘cabal
>> update’ and know about exactly the same package versions available but
>> end up with different version numbers used because one ‘cabal update’d
>> before the info on Hackage was changed and the other did not.
>>
>> But John is right that sometimes making releases is tiresome. For
>> version bumps it can be as easy as checking out the last tag, bumping
>> deps and uploading, but it can be much more involved too. I admit that I
>> used this myself, although only for the stuff that hope to release
>> properly soon and I happen to maintain on NixOS too which AFAICT is the
>> only distro which packages Haskell fast enough to notice breakage within
>> few days (in huge part thanks to Peter Simons ;) )
>>
>> So clearly there is a problem here: only cabal users use the information
>> from Hackage info bumps. Everyone else who looks into the tarball just
>> fails to build. cabal users who happen to cabal update at unfortunate
>> time also fail to build.
>>
>> I can only think of one ‘solution’ which fixes this and doesn't require
>> considerable effort and co-ordination between maintainers and packagers.
>> Here is how situation currently looks like
>>
>> 1. I upload awesomepackage (ap) to Hackage under ap-1.0.0.0
>> 2. Time passes, I have commits on top
>> 3. One of those packager type of folk who like to demand things come
>> onto my issue tracker and say I should update my dependencies but I
>> don't want to release right now
>> 4. I go onto Hackage and use that new cool feature, changing the cabal
>> file on ap-1.0.0.0.
>> 5. Packager is unhappy because the problem is not fixed, ap-1.0.0.0
>> gives them the same thing. The only thing they can do is monkeypatch
>> deps on their side.
>>
>> Instead, I think at steps 4 and 5 the following should happen:
>>
>> 4. I make changes to ap-1.0.0.0, Hackage generates ap-1.0.0.0-fix1
>> package with identical content but new info without any extra effort on
>> my side.
>> 5. Packager simply now points to ap-1.0.0.0-fix1 which has the new hash
>> and builds great again. cabal does The Right Thing™ here and resolves
>> ap-1.0.0.0 to ap-1.0.0.0-fix1 so everything keeps working there as it
>> does now.
>>
>> It does seem a bit wasteful space-wise but with clever programming on
>> Hackage side that could probably be avoided. There is no longer a
>> problem with these ‘hidden’ changes happening. Everyone can point at the
>> *exact* version they want used like they do now and they get what
>> exactly what they asked for. A less wasteful way would be for the
>> packaging distros to update their tools to look at the Hackage-only
>> cabal files as well. Personally I think that seems like the wrong place
>> to solve this though.
>>
>> Lastly, Hackage should announce with big fat red letters on the front
>> page of the package that the source tarball contains different info than
>> what's shown on Hackage. It's really not possible to tell as a user at
>> the moment unless we specifically look for it.
>>
>> Sorry for the long-winded message, hopefully you heeded the TL;DR at the
>> top and skipped this.
>>
>>
>
> --
> Andreas Abel  <><      Du bist der geliebte Mensch.
>
> Department of Computer Science and Engineering
> Chalmers and Gothenburg University, Sweden
>
> andreas.abel at gu.se
> http://www2.tcs.ifi.lmu.de/~abel/
>
> _______________________________________________
> Haskell-Cafe mailing list
> Haskell-Cafe at haskell.org
> http://www.haskell.org/mailman/listinfo/haskell-cafe
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/haskell-cafe/attachments/20141028/bcf4ee56/attachment.html>