[Haskell-cafe] A new cabal odissey: cabal-1.8 breaking its own neck by updating its dependencies

Sat Sep 11 18:39:43 EDT 2010

Hi all,
sadly, I can't post all details.

I just got a kernel panic and only parts of the detailed log were
saved - my bad (but this is something that kind-of never happens,
except, by Murphy's law, now). I could try reproducing it another
time, but it took some time, so for now I'll rely on my memory.

On Sat, Sep 11, 2010 at 20:56, Thomas DuBuisson
<thomas.dubuisson at gmail.com> wrote:
>> - when recompiling a package with ABI changes, does cabal always
>> update dependent packages?
>
> If Foo depends on Bar and there is a new version of Foo that specifies
> a newer version of Bar then yes, the newer library being depended on
> will be build too (out of necessity).

> OTOH, if you are building a new version of a package on which others
> depend it won't build all the others.  Ex: build a new "containers"
> package doesn't cause any of the ~1400 packages depending on it to be
> rebuilt.

However, the old containers package stays there.

In my case, the same version of a package (old-time) was recompiled
against a different version of one of its dependencies (old-library),
and that broke the ABI. That's tricky. And Cabal recompiled most of
the package which needed help, except Cabal itself.

>> It seems "not always" - it didn't update
>> itself, nor refuse the breaking upgrade,

> I don't really know what "it" is.  Something to do with recompiling
> Cabal and cabal-install I take it, but I'll refrain from comment
> seeing as I don't understand what you're doing.

My bad - I wrote "it didn't update itself", but it would have been
more precise to write "Cabal did not recompile itself". And probably
also cabal-install should have been recompiled.

In my case, I got cabal proposing the following change while I was
trying to install pandoc-1.5:

$ cabal install -v pandoc-1.5
[snip, see attached inst-pandoc-1.5-without-pref-old-loc-_1.txt for
full output.]
old-time-1.0.0.2 (reinstall) changes: old-locale-1.0.0.1 -> 1.0.0.2
[snip]

When I did that, the result was that a symbol from old-time,
referenced from the Cabal package, disappeared, preventing further
package builds. It was possible because the symbol was
oldzmtimezm1zi0zi0zi2_SystemziTime_a149_{closure,info}, i.e. an
anonymous function. I wouldn't be surprised if the "a149" part were
randomly generated, causing the ABI breakage at every rebuild. But
recompiling against another version of a dependency is surely enough
(I guess that if inlining is disabled these things can't happen).
The net result, when Cabal was compiling the main Setup.hs of another
package (texmath IIRC), was a linker error on that symbol. Luckily,
the old version of old-time was in the system DB, so I could just
uninstall the new version from the user DB (after I decoded the linker
error, which wasn't trivial even knowing more about linking than one
would want to know).

An interesting fact is that cabal had recompiled many other packages
depending on old-time (HTTP, directory, process, zip-archive), just
not Cabal.

Maybe recompiling Cabal could have fixed it, but I avoided trying, and
reverted everything. The interesting part is that one would need to
recompile it after building the new version of its dep ("old-time"
here) but before installing it in place of the old binary, since after
reinstalling old-time-1.0.0.2 and before installing the new Cabal, no
package can be installed, including Cabal itself.
That's rather tricky to do it, and it can't be done by hand. And cabal
is probably unique in having this need - I've used CPAN, Gentoo's
portage, and so on, and for various reasons (stable ABIs, no
compilation going on, etc.) this scenario can't probably happen.

After adding "preference: old-locale == 1.0.0.1", the output of $
cabal install -v pandoc-1.5 became the one in
inst-pandoc-1.5-with-pref-old-loc-_1.txt, where no upgrade of
old-time/old-locale is needed. Interestingly, HTTP, directory,
process, zip-archive were not reinstalled, which confirms that Cabal
had reinstalled them before just because of an upgrade to the
dependencies.

I managed to successfully remove old-locale-1.0.0.2 from my system and
recompile all broken dependencies, even if it was rather tricky - even
when ghc-pkg reported no errors, several dependencies were still
broken, and the error message was crazy (sadly, I lost the log).

>> While package
>> removal is not supported through cabal, it is sometimes needed

> Why?  What I see is a need for users to understand ghc-pkg (or
> whatever package management tool exists for their Haskell compiler).
> Should "cabal uninstall" provide a unified interface to some number of
> Haskell compiler packaging systems?  It could but doesn't seem like a
> priority.

The frontend stuff is not a priority, but my point was not who should
support uninstallation, but that it is necessary to support it, and
that furthermore:

>>A related idea is that Gentoo had a tool which not only extracted such
>>dependencies, but recompiled all affected packages.

If uninstalling packages replaced by other versions is supported at
all, like on Gentoo, such a tool is important for Haskell, too. And
uninstalling packages is sometimes needed. It is conceivable that
after upgrading a package I want to get rid of the old version. In my
case, I wanted to uninstall a package to "downgrade" it, because it
caused breakage; and I wanted to get its dependencies recompiled. This
is especially important since Haskell libraries don't have a stable
ABI.

Additionally, I could have a security breach in a Haskell library and
want to guarantee an upgrade - not all vulnerabilities are C-specific
(take cross-site scripting, SQL injection, bad handling of temporary
files for setuid programs, bad validation of user input for a mail
server...). Like this:
http://www.amateurtopologist.com/2010/04/23/security-vulnerability-in-haskell-with-cgi/

ghc-pkg doesn't uninstall packages, it just deregisters them and
doesn't remove the files. Indeed, there's already a ticket open for
having 'cabal uninstall':

http://hackage.haskell.org/trac/hackage/ticket/234

I should just try adding a ticket for the above idea.
-- 
Paolo Giarrusso - Ph.D. Student
http://www.informatik.uni-marburg.de/~pgiarrusso/
-------------- next part --------------
/usr/bin/ghc --numeric-version
looking for package tool: ghc-pkg near compiler in /usr/bin
found package tool in /usr/bin/ghc-pkg
/usr/bin/ghc-pkg --version
/usr/bin/ghc --supported-languages
Reading installed packages...
/usr/bin/ghc-pkg dump --global
/usr/bin/ghc-pkg dump --user
/usr/bin/ghc --print-libdir
Reading available packages...
Resolving dependencies...
selecting pandoc-1.5 (hackage) and discarding HTTP-3000.0.0, 3001.0.0,
3001.0.1, 3001.0.2, 3001.0.3, 3001.0.4, 3001.1.3, 3001.1.4, 3001.1.5,
4000.0.0, 4000.0.1, 4000.0.2, 4000.0.3, 4000.0.4, filepath-1.0, mtl-1.0,
parsec-2.0, 3.0.0, 3.0.1, 3.1.0, utf8-string-0.1, 0.2, xml-1.2.6, 1.3.1,
1.3.2, 1.3.3, 1.3.4, zip-archive-0.1, 0.1.1, 0.1.1.1, 0.1.1.2 and 0.1.1.3
selecting zip-archive-0.1.1.6 (installed or hackage) and discarding
bytestring-0.9, utf8-string-0.3, zip-archive-0.1.1.4 and 0.1.1.5
selecting texmath-0.3.0.2 (installed or hackage) and discarding texmath-0.1,
0.1.0.1, 0.1.0.2, 0.1.0.3, 0.1.0.4, 0.1.1, 0.2, 0.2.0.1, 0.2.0.2, 0.2.0.3,
0.2.0.4, 0.3 and 0.3.0.1
selecting utf8-string-0.3.5 (installed or hackage) and discarding
utf8-string-0.3.1, 0.3.1.1, 0.3.2, 0.3.3, 0.3.4 and 0.3.6
selecting xml-1.3.7 (installed or hackage) and discarding xml-1.3.5
selecting zlib-0.5.2.0 (installed or hackage) and discarding zlib-0.2, 0.3,
0.4, 0.4.0.1, 0.4.0.2, 0.4.0.3, 0.4.0.4 and 0.5.0.0
selecting digest-0.0.0.8 (installed or hackage) and discarding digest-0.0.0.1,
0.0.0.2, 0.0.0.3, 0.0.0.4, 0.0.0.5, 0.0.0.6 and 0.0.0.7
selecting xhtml-3000.2.0.1 (installed or hackage) and discarding
xhtml-3000.0.0, 3000.0.1, 3000.0.2.1, 3000.0.2.2, 3000.1.0.0 and 3000.2.0.0
selecting binary-0.5.0.1 (installed or hackage) and discarding binary-0.2,
0.3, 0.4, 0.4.1, 0.4.2, 0.4.3, 0.4.3.1, 0.4.4, 0.5 and 0.5.0.2
selecting extensible-exceptions-0.1.1.0 (installed or hackage) and discarding
extensible-exceptions-0.1.0.1, 0.1.1.1 and 0.1.1.2
selecting HTTP-4000.0.9 (installed or hackage) and discarding HTTP-4000.0.5,
4000.0.6, 4000.0.7 and 4000.0.8
selecting network-2.2.1.4 (installed or hackage) and discarding network-2.0,
2.1.0.0, 2.2.0.0, 2.2.0.1, 2.2.1, 2.2.1.1, 2.2.1.2, 2.2.1.3, 2.2.1.5, 2.2.1.6
and 2.2.1.7
selecting parsec-2.1.0.1 (installed or hackage) and discarding parsec-2.1.0.0
selecting mtl-1.1.0.2 (installed or hackage) and discarding mtl-1.1.0.0,
1.1.0.1 and 1.1.1.0
selecting process-1.0.1.1 (installed or hackage) and discarding
process-1.0.0.0, 1.0.1.2 and 1.0.1.3
selecting pretty-1.0.1.0 (installed or hackage) and discarding pretty-1.0.0.0
and 1.0.1.1
selecting directory-1.0.0.3 (installed or hackage) and discarding
directory-1.0.0.0, 1.0.1.0, 1.0.1.1 and 1.0.1.2
selecting unix-2.3.2.0 (installed or hackage) and discarding unix-2.0,
2.2.0.0, 2.3.0.0, 2.3.1.0, 2.4.0.0, 2.4.0.1 and 2.4.0.2
selecting bytestring-0.9.1.4 (installed or hackage) and discarding
bytestring-0.9.0.1, 0.9.0.2, 0.9.0.3, 0.9.0.4, 0.9.1.0, 0.9.1.1, 0.9.1.2,
0.9.1.3, 0.9.1.5, 0.9.1.6 and 0.9.1.7
selecting ghc-prim-0.1.0.0 (installed)
selecting rts-1.0 (installed)
selecting filepath-1.1.0.2 (installed or hackage) and discarding
filepath-1.1.0.0, 1.1.0.1, 1.1.0.3 and 1.1.0.4
selecting old-time-1.0.0.2 (installed or hackage) and discarding
old-time-1.0.0.0, 1.0.0.3, 1.0.0.4 and 1.0.0.5
selecting old-locale-1.0.0.2 (installed or hackage) and discarding
old-locale-1.0.0.0 and 1.0.0.1
selecting containers-0.2.0.1 (installed or hackage) and discarding
containers-0.1.0.0, 0.1.0.1, 0.2.0.0 and 0.3.0.0
selecting array-0.2.0.0 (installed or hackage) and discarding array-0.1.0.0,
0.3.0.0 and 0.3.0.1
selecting base-4.1.0.0 (installed) and 3.0.3.1 (installed) and discarding
syb-0.1.0.0, 0.1.0.1, 0.1.0.2, 0.1.0.3, 0.2 and 0.2.1
selecting integer-0.1.0.1 (installed)
selecting syb-0.1.0.1 (installed)
In order, the following would be installed:
old-time-1.0.0.2 (reinstall) changes: old-locale-1.0.0.1 -> 1.0.0.2
HTTP-4000.0.9 (reinstall)
directory-1.0.0.3 (reinstall)
process-1.0.1.1 (reinstall)
zip-archive-0.1.1.6 (reinstall)
pandoc-1.5 (new version)
-------------- next part --------------
/usr/bin/ghc --numeric-version
looking for package tool: ghc-pkg near compiler in /usr/bin
found package tool in /usr/bin/ghc-pkg
/usr/bin/ghc-pkg --version
/usr/bin/ghc --supported-languages
Reading installed packages...
/usr/bin/ghc-pkg dump --global
/usr/bin/ghc-pkg dump --user
/usr/bin/ghc --print-libdir
Reading available packages...
Resolving dependencies...
selecting pandoc-1.5 (hackage) and discarding HTTP-3000.0.0, 3001.0.0,
3001.0.1, 3001.0.2, 3001.0.3, 3001.0.4, 3001.1.3, 3001.1.4, 3001.1.5,
4000.0.0, 4000.0.1, 4000.0.2, 4000.0.3, 4000.0.4, filepath-1.0, mtl-1.0,
parsec-2.0, 3.0.0, 3.0.1, 3.1.0, utf8-string-0.1, 0.2, xml-1.2.6, 1.3.1,
1.3.2, 1.3.3, 1.3.4, zip-archive-0.1, 0.1.1, 0.1.1.1, 0.1.1.2 and 0.1.1.3
selecting zip-archive-0.1.1.6 (installed or hackage) and discarding
bytestring-0.9, utf8-string-0.3, zip-archive-0.1.1.4 and 0.1.1.5
selecting texmath-0.3.0.2 (installed or hackage) and discarding texmath-0.1,
0.1.0.1, 0.1.0.2, 0.1.0.3, 0.1.0.4, 0.1.1, 0.2, 0.2.0.1, 0.2.0.2, 0.2.0.3,
0.2.0.4, 0.3 and 0.3.0.1
selecting utf8-string-0.3.5 (installed or hackage) and discarding
utf8-string-0.3.1, 0.3.1.1, 0.3.2, 0.3.3, 0.3.4 and 0.3.6
selecting xml-1.3.7 (installed or hackage) and discarding xml-1.3.5
selecting zlib-0.5.2.0 (installed or hackage) and discarding zlib-0.2, 0.3,
0.4, 0.4.0.1, 0.4.0.2, 0.4.0.3, 0.4.0.4 and 0.5.0.0
selecting digest-0.0.0.8 (installed or hackage) and discarding digest-0.0.0.1,
0.0.0.2, 0.0.0.3, 0.0.0.4, 0.0.0.5, 0.0.0.6 and 0.0.0.7
selecting xhtml-3000.2.0.1 (installed or hackage) and discarding
xhtml-3000.0.0, 3000.0.1, 3000.0.2.1, 3000.0.2.2, 3000.1.0.0 and 3000.2.0.0
selecting binary-0.5.0.1 (installed or hackage) and discarding binary-0.2,
0.3, 0.4, 0.4.1, 0.4.2, 0.4.3, 0.4.3.1, 0.4.4, 0.5 and 0.5.0.2
selecting extensible-exceptions-0.1.1.0 (installed or hackage) and discarding
extensible-exceptions-0.1.0.1, 0.1.1.1 and 0.1.1.2
selecting HTTP-4000.0.9 (installed or hackage) and discarding HTTP-4000.0.5,
4000.0.6, 4000.0.7 and 4000.0.8
selecting network-2.2.1.4 (installed or hackage) and discarding network-2.0,
2.1.0.0, 2.2.0.0, 2.2.0.1, 2.2.1, 2.2.1.1, 2.2.1.2, 2.2.1.3, 2.2.1.5, 2.2.1.6
and 2.2.1.7
selecting parsec-2.1.0.1 (installed or hackage) and discarding parsec-2.1.0.0
selecting mtl-1.1.0.2 (installed or hackage) and discarding mtl-1.1.0.0,
1.1.0.1 and 1.1.1.0
selecting process-1.0.1.1 (installed or hackage) and discarding
process-1.0.0.0, 1.0.1.2 and 1.0.1.3
selecting pretty-1.0.1.0 (installed or hackage) and discarding pretty-1.0.0.0
and 1.0.1.1
selecting directory-1.0.0.3 (installed or hackage) and discarding
directory-1.0.0.0, 1.0.1.0, 1.0.1.1 and 1.0.1.2
selecting unix-2.3.2.0 (installed or hackage) and discarding unix-2.0,
2.2.0.0, 2.3.0.0, 2.3.1.0, 2.4.0.0, 2.4.0.1 and 2.4.0.2
selecting bytestring-0.9.1.4 (installed or hackage) and discarding
bytestring-0.9.0.1, 0.9.0.2, 0.9.0.3, 0.9.0.4, 0.9.1.0, 0.9.1.1, 0.9.1.2,
0.9.1.3, 0.9.1.5, 0.9.1.6 and 0.9.1.7
selecting ghc-prim-0.1.0.0 (installed)
selecting rts-1.0 (installed)
selecting filepath-1.1.0.2 (installed or hackage) and discarding
filepath-1.1.0.0, 1.1.0.1, 1.1.0.3 and 1.1.0.4
selecting old-time-1.0.0.2 (installed or hackage) and discarding
old-time-1.0.0.0, 1.0.0.3, 1.0.0.4 and 1.0.0.5
selecting old-locale-1.0.0.1 (installed or hackage) and discarding
old-locale-1.0.0.0 and 1.0.0.2
selecting containers-0.2.0.1 (installed or hackage) and discarding
containers-0.1.0.0, 0.1.0.1, 0.2.0.0 and 0.3.0.0
selecting array-0.2.0.0 (installed or hackage) and discarding array-0.1.0.0,
0.3.0.0 and 0.3.0.1
selecting base-4.1.0.0 (installed) and 3.0.3.1 (installed) and discarding
syb-0.1.0.0, 0.1.0.1, 0.1.0.2, 0.1.0.3, 0.2 and 0.2.1
selecting integer-0.1.0.1 (installed)
selecting syb-0.1.0.1 (installed)
In order, the following would be installed:
pandoc-1.5 (new version)