[Haskell-cafe] Hackage 2 maintainership

[Duncan Coutts]

Hi Ben,

On 13 February 2012 23:44, Ben Gamari <bgamari.foss at gmail.com> wrote:
> Hey all,
>
> Those of you who follow the Haskell subreddit no doubt saw today's post
> regarding the status of Hackage 2. As has been said many times in the
> past, the primary blocker at this point to the adoption of Hackage 2
> appears to be the lack of an administrator.

Yes, much of it is lack of an individual to keep momentum up and keep
everyone else motivated. While I'm keen that hackage moves forward, my
volunteer time is spread too thin to be that person keeping everything
organised. That said, where I spend my volunteer time is to a large
part directed by what other people are doing, it's much more fun and
motivating if there's other people working with you.

Speaking of which, I spent much of this evening fixing things, more
details below.

> It seems to me this is a poor reason for this effort to be held
> up. Having taken a bit of time to consider, I would be willing to put in
> some effort to get things moving and would be willing to maintain the
> haskell.org Hackage 2.0 instance going forward if necessary.

That would be great. So in the short term I'm very happy to get the
help and in the longer term I'm happy to hand over to anyone sensible
who puts in the effort. That person could be you, someone else or a
team of several people.

More immediately, my general policy with commit access is to give it
to anyone who's sent a few good patches. Currently there are 7 people
with write access to the darcs repo on code.h.o. It is of course also
fine for people to maintain their own public branches (which they can
do using git or darcs, whichever).

> I currently have a running installation on my personal machine and
> things seem to be working as they should. On the whole, installation was
> quite trivial, so it seems likely that the project is indeed at a point
> where it can take real use (although a "logout" option in the web
> interface would make testing a bit easier).

Yes, we're at the stage where we can run a public testing instance.
You'll see there's a bit more to implement and test for a switchover.

> That being said, it would in my opinion be silly to proceed without
> fixing the Hackage trac. It was taken down earlier this year due to
> spamming[1] and it seems the recovery project has been orphaned. I would
> be willing to help with this effort, but it seems like the someone more
> familiar with the haskel.org infrastructure might be better equipped to
> handle the situation.

I spent a couple hours on this this evening and I've finally fixed it
(I hope). I still need to purge a bit of wiki/ticket spam (help
apreciated there). Sadly I've had to blow away the previos login
accounts, but I've semi-restored by copying the ghc trac accounts. So
if you happen to have an account on the ghc trac, then your login
should work for the hackage trac. Otherwise you'll need to re-register
as if it was a new account.

> It seems that this process will go something like this,
>  1) Bring Hackage trac back from the dead

Check.

>  2) Bring up a Hackage 2 instance along-side the existing
>     hackage.haskell.org

Yes, now that the trac is back, you can see what notes we have on the
switchover process at:
http://hackage.haskell.org/trac/hackage/wiki/HackageDB/2.0

Note also that the nice people at factisresearch.com have given us a
VM with enough memory (8GB) for the purpose of running a public test
with the full package set (in principle it should not need so much
memory, but we currently keep unnecessary package metadata in memory).

So thanks to you and others this evening motivating me, I've also
taken Max's latest patches to my tar package (which coincidentally I
released yesterday) and the corresponding hackage-server patch and set
it running at:

http://hackage.factisresearch.com/

This is running the latest upstream darcs version. I have also fired
off a one-shot mirroring operation. This will mirror all the existing
packages from hackage. It'll probably take half a day or so to
complete since there's something like 30-40k tarballs to be copied
over. I'll check the logs tomorrow hopefully and after that kick off a
live/continuous mirror so it'll get new updates from the main hackage
within 20-30 min or so.

Last time Max and I tried this we were able to mirror almost all
packages. Most of the unmirrorable ones at the time were due to
packages with quirks in their tar format, which is what his tar
patches were aimed at. So I'm hopeful we'll now have only a tiny
handful of unmirrorable packages.

>  3) Enlist testers
>  4) Let things simmer for a few weeks/months ensuring nothing explodes
>  5) After it's agreed that things are stable, eventually swap the
>     Hackage 1 and 2 instances

Right, that's more or less it. Other details on the wiki (and if
there's anything missing, edit it).

You'll see that there are still some missing components. In particular
while I finished the live mirroring client, and Max has done a doc
builder client, I think we're still missing a normal build client (one
would start with Max's doc client).

> This will surely be a non-trivial process but I would be willing to move
> things forward.

Taking an organisational / management / cheerleader role would be
imensely useful (as well as technical obviously). I can help because I
know more or less what needs to be done and I'm very happy to answer
questions, grant push access etc.

Duncan