[arch-haskell] Re: Hackage support in bauerbill & follow-up on cabal2arch

[Don Stewart]

xyne:
> Hi,
> 
> I've added preliminary support for Hackage to Bauerbill. The following
> now work:
> 
> Btw, for now at least, if you enable AUR support as well (--aur) then
> AUR packages take precedence over Hackage packages. The reasoning is
> that AUR packages are more likely to be consistent as they're managed
> by arch-haskell and thus have some human oversight. This may change
> later if it's deemed move useful to give precedence to Hackage when
> both are enabled.

I think it is reasonable to prefer, in order,

    [extra]
    [community]
    AUR
    Hackage

in particular, AUR is likely to have any special cases handled that
cabal2arch couldn't handle (which sometimes happens, though rarely).

> I've updated to the latest version of cabal2arch and I've noticed that
> most of the clutter is gone. I appreciate the changes and consider
> them to be a great improvement, but it still creates an archive of the
> build directory which I neither need nor want.
> 
> I would like to make the following suggestions:
> 
> *) add an option to create the archive of the build directory and
> suppress it by default so that cabal2arch only create the build dir
> with the PKGBUILD and install file when run with no options
> 
> *) add an option to create the PKGBUILD and install file in the current
> working directory, and create nothing else
> 
> *) enable batching of multiple cabal files, so that it can accept
> multiple URLs and paths on the command line (including paths to
> directories which contain multiple cabal files)

These all look reasonable. I've added them to my todo.
  
> 
> I've also noticed that it downloads source files from Hackage. Is that
> just to get the checksums for the archive? If so, it seems rather
> wasteful. Would it not be possible to provide that information in the
> cabal file or via a web API on Hackage?

It is indeed only for the md5. We could store it on Hackage. I've filed
a ticket for Hackage to do this.

    http://hackage.haskell.org/trac/hackage/ticket/635

We could also just compute our own list of hashes once and store them (I
could add support for such a thing to cabal2arch I suppose).


> If it is not possible to create the PKGBUILD without the source
> archive, please add an option to first check if the source archive
> exists in the build directory and mirror it if it doesn't. This would

That's a good idea. Currently cabal2arch checks for a .cabal file in the
current directory, not the .tar.gz that it would download.

> prevent redundant downloads when building packages (which saves time
> and bandwidth for all involved). Bauerbill in particular would benefit
> from this by being able to handle the downloads itself (e.g. in
> parallel). It could also preserve them between build sessions and
> mirror them to only update when necessary.
> 
> Apropos the availability of information from Hackage, the Hackage
> backend of Bauerbill currently uses the log and pkg-list.html
> (in /packages/archives/) to create a plain-text database of all packages
> on Hackage. You can view the "database" file created by bauerbill
> (/tmp/bauerbill/hackage/hackage_db by default). This file is smaller
> than both the log and pkg-list.html and follows the line format
> <name> <version> <category> <type>: <synopsis>
> 
> Would it be possible to generate this file directly on Hackage? Again,
> this would save bandwidth and it would also enable control over the
> versions built by Bauerbill. Currently it tracks the latest versions by
> simple parsing of the log. If the database file were instead provided
> then it would use whichever versions it contained. This would make it
> possible to specify which versions should be built. I don't know if
> this would be an issue but version compatibility does seem to come up
> quite often.


There's a much easier way -- since the database is already mirroed
locally by cabal. The cabal-install tool holds this database,
and can list it for you:

    $ cabal list

And we have a programmatic binding to it in the archlinux package:

    http://hackage.haskell.org/packages/archive/archlinux/0.3/doc/html/Distribution-ArchLinux-Report.html#v%3AloadPackageIndex 

which returns a table of all packages and their latest versions.


> One final niggle: cabal2arch doesn't understand relative paths to
> parent directories, e.g. "../cabal/foo.cabal".

Added.
  
> p.s. Would it be useful to be able to rebuild all Hackage packages with
> a simple command? If cabal2arch placed Hackage packages in a group, I
> could easily add a "--rebuild-group" option to Bauerbill. I'm asking
> because I don't actually know if this would be useful

Mmmm. Currently I just loop over all the packages (computed as follows,
based on the AUR maintainer):

    import Distribution.ArchLinux.AUR
    import Control.Monad

    me = "arch-haskell"

    main = do
        packages <- maintainer me
        forM_ packages $ putStrLn . packageName


Hackage itself can be built with the cabal plan, assuming you have a
coherent set of packages. We use this for regression testing against
GHC.

Thanks for all the feature requests. I've taken action on some, and
added todos for others.

-- Don