[arch-haskell] State of Affairs: Summarizing 83 days worth of experience

[Peter Simons]

Hi guys,

it's been 83 days since I've begun to actively contribute to the
ArchHaskell project. During that period, I've maintained our habs tree.
I've seen to it that AUR is in sync with that tree. I've made an effort
to compile a repository of binary packages, and I've also addressed all
kinds of user feedback and bug reports. Based on that experience, I'd
like to take a step back and summarize my overall impression of where
this projects stands today.

First of all, I believe that our joint efforts have been very worthwhile
and successful. habs is in better shape today than it was before. To the
best of my knowledge, none of the PKGBUILDs that we publish is outright
broken, which is a *major* improvement. Secondly, ArchLinux now complies
to the Haskell Platform, which also feels like an important achievement.
Furthermore, a fairly large number of small annoyances have been
addressed, i.e. we have consistent support for shared linking, our
generated PKGBUILD files have been improved in terms of both style and
substance, and we've taken on the habit of utilizing $pkgrel to trigger
necessary re-builds, which is something our users had to figure out on
their own in the past. Given all that, I think we can be a little proud.

At the same time, I feel a little pessimistic about the future of our
endeavor. When I said "habs" in the previous paragraph, I really meant a
subset of habs worth 118 packages that I actively maintain, namely those
packages that I personally care about. I didn't even try to keep
packages up-to-date that I don't personally care about. Hackage,
however, features a whopping total of 2670 packages, which means that
I've been actively maintaining less than 5% of Hackage!

Now, based on my experience from maintaining 5% of Hackage, I've arrived
at the conclusion that our tools and our procedures are quite inadequate
to get the job done. Maintaining 118 packages in an orderly fashion has
been really difficult. I wouldn't dream of even trying to maintain 2500+
packages that way. The most significant problems I've encountered are:

 1) A lack of coordination. We have made a couple of attempts to define
    policies, procedures, and responsibilities for this project, but
    ultimately we never really got anywhere. The net result is that
    everyone of us is working on whatever interests him, but the others
    mostly don't even know about it. Consequently, we are inefficient.
    For instance, we develop patches for the ArchLinux library, and when
    they're ready, we throw them away, because we realize way too late
    that we don't like what they do. Similarly, we perform updates in
    [extra], and then we revert them again, because we notice way too
    late that they break someone else's efforts. It feels like most of
    the things we've accomplished were being accomplished though
    individual machismo, rather than through a coordinated team effort.

 2) Lack of communication. In my experience, it's incredible hard to get
    any sort of response from those of us who maintain [extra] and
    [community]. The packages in those repositories are the foundation
    on which everything else has to be built. Yet, the packages in those
    repositories tend to be modified without any consultation or advance
    warning. In some cases, changes in [extra] broke compilation of
    other habs packages -- such as cabal2arch --, but for the longest
    time it was impossible to get anyone to care about that. On the
    other hand, I've repeatedly asked for packages in [extra] to be
    updated, because I needed newer versions to build other packages in
    habs, but those updates just don't happen. Maybe there are good
    technical reasons why those updates don't happen, but that's beside
    the point. The point is that I don't know anything about it, and I
    don't know how to find out either.

 3) Inadequate tools. The cabal2arch utility is a great, but in its
    current shape it cannot re-generate the habs in a fashion that I'd
    call "automatic". For some packages, such as OpenGL or GLUT, the
    cabal2arch output is outright broken and doesn't compile. For other
    packages, such as cabal2arch itself, the generated PKGBUILD compiles
    but is incorrect anyway, because run-time dependencies are declared
    as being build-time dependencies. There are several other packages
    that cannot be generated with cabal2arch, such as haskell-platform,
    and that need manual editing before they can be published. So far,
    we have been addressing these problems in a way that feels "ad hoc",
    to put it mildly. Being a professional software engineer, that makes
    me quite uncomfortable, because I have the impression that we don't
    fully understand what it is that we're trying to do.

    Furthermore, we seem to have no functioning tools that help us
    automate the updating process. My experience so far is that even the
    tiniest trivial update has a significant potential to break the
    build of dozens of other packages. Basically, there seem to be
    trivial updates in Haskell land. Whenever such breakage occurs,
    there is no easy way to figure out how to remedy the version
    conflicts. Doing that manually is quite an effort when 118 packages
    ought to compile, but remedying those conflicts manually for 2500+
    packages is not going to be possible.

In my humble opinion, we have to figure out how to improve our team work
if we want to succeed at providing a comprehensive set of Haskell
packages for ArchLinux.

Take care,
Peter