[arch-haskell] State of Affairs: Summarizing 83 days worth of experience

[Magnus Therning]

On Sun, Jan 9, 2011 at 23:02, Peter Simons <simons at cryp.to> wrote:
[...]
>> In your initial email you say things are broken, but in my opinion you
>> don't provide enough information for me to understand WHY you say that.
>
> I am sorry, but it strikes me as fairly unreasonable to summarize my
> original posting as me saying "thinks are broken". I was providing slightly
> more information than that. Surely you are exaggerating a bit?

Of course I exaggerate, and not only a bit, a whole lot.

> After all,
> you yourself said that my feedback was "invaluable". Now you seem to be
> saying that my posting wasn't worth much. How is it possible that you have
> changed your opinion so dramatically?

To be very blunt, I said your EXPERIENCE is invaluable.  In your
initial email you didn't recount that experience in a way that clearly
pointed out what wasn't working.  That means your initial email wasn't
invaluable, because there was nothing in it that I could act on.

>
>  >>  1) A lack of coordination. We have made a couple of attempts to define
>  >>     policies, procedures, and responsibilities for this project, but
>  >>     ultimately we never really got anywhere. The net result is that
>  >>     everyone of us is working on whatever interests him, but the others
>  >>     mostly don't even know about it. Consequently, we are inefficient.
>  >
>  > This is true to a large extent.  What do you feel is left to define?
>
> Most importantly, we should agree on a common goal that we are trying to
> achieve. Personally, my goal has been to provide up-to-date ArchLinux build
> instructions for the Haskell packages published on Hackage. To achieve that
> goal, I maintained our habs tree. Based on that habs tree, I felt that it's
> feasible to maintain both AUR and a binary package repository, which was my
> secondary goal. I have kind-of achieved both of these goals, but only when
> "Hackage" is defined as "5% of Hackage". So if those goals ought to be
> achieved completely, then we need a strategy how to deal with the remaining
> 95% of Hackage. This realization compelled me to start this thread.
>
> I notice that you didn't commit much to habs, so clearly you had different
> goals. I wonder what those were? What exactly is it that you are trying to
> achieve?

My goals were largely the same, my available time wasn't.  I had also
hoped to contribute more to the improvement of our tools.

>  >>     We develop patches for the ArchLinux library, and when they're
>  >>     ready, we throw them away, because we realize way too late that we
>  >>     don't like what they do.
>  >
>  > I don't think this is that uncommon in FOSS when people work on their own
>  > without telling people what they are doing.
>
> Yes, of course you are right. But the point of my remark wasn't that this
> kind of thing would be uncommon. Rather, I complained about the fact that it
> is highly inefficient. We are 3(!) people trying to maintain a set of almost
> 2000 packages. How are we supposed to accomplish that when we waste our time
> writing patches that are ultimately thrown away?

I agree fully with you on this.  However, I don't share your level of
concern, I just don't feel that there has been many patches that in
the end have been thrown away.

> We are volunteers doing this project in our spare time, so it's obviously
> perfectly alright that each of us does whatever he feels like. I'm not
> trying to tell anyone what they are supposed to do or don't do. However,
> unless we coordinate our efforts to some extend, we cannot possibly achieve
> the goal of providing Hackage to ArchLinux users. Based on the assumption
> that this is our common goal, it seems obvious to me that we need to
> coordinate our efforts more than we did in the past.

[...]

>  >>  3) Inadequate tools. The cabal2arch utility is a great, but in its
>  >>     current shape it cannot re-generate the habs in a fashion that I'd
>  >>     call "automatic".
>  >
>  > Have you raised bugs for the individual issues here, so that we have them
>  > documented in a central place?
>
> Yes, of course I did. Didn't you receive notifications from Github?

Where are those bugs?

I did some bugs raised by you on the cabal2arch project, but they were
all package specific, i.e. "cabal2arch doesn't generate correct output
for Foo".  These issues say that there are bugs in cabal2arch, they
don't say that cabal2arch is inadequate for maintaining 2000 packages
in AUR.

>  >>     Furthermore, we seem to have no functioning tools that help us
>  >>     automate the updating process. My experience so far is that even the
>  >>     tiniest trivial update has a significant potential to break the
>  >>     build of dozens of other packages. Basically, there seem to be
>  >>     trivial updates in Haskell land.
>  >
>  > What process are you using? Both when adding completely new packages and
>  > when puling in a new version of a package. What tools are you currently
>  > using? What actions are manual, and what actions are only partially
>  > covered by tools?
>
> Well, as I said, we don't have any functioning tools or procedures that
> assist in process of updating a package. So I'm not using *any* tools, and
> *all* actions are manual. It feels like you expect me to provide extensive
> formal documentation of what I've been doing in the last 3 months. I am
> sorry, but I don't have the time to submit a report about everything.

Oh, no, I don't expect that at all.  I wouldn't read it if you wrote
it anyway ;-)

Here's my "process" for updating packages on AUR:

1. Pull down any changes to my local copy of the HABS tree.
2. Open a clean chroot and bind-mount HABS and some personal tools into it.
3. Get the URL for the package that needs updating.
4. Run `cabal2arch <URL>` in the HABS top-level.
5. Switch to the chroot and build the updated package and it's
dependencies (I have a simple shell script that does this)
6. If the build fails, then  revert the changes to the package in HABS.
7. Go back to 2 and update another package.
8. When no more packages need updating, build the source packages for
all modified packages.
9. Use aurploader to upload the source packages to AUR.

The problem with this is that step 5 can take a VERY long time, and
sometimes I end up doing unnecessary work because there's no easy way
of ordering the packages I'm updating such that the number of builds
are minimised.

Does this process match what you do?
Maybe you have more effective ways of performing some of these steps
that I could benefit from?

> Instead, I recommend that you just try to do a few updates yourself. Then
> you'll find out what the problems are. Frankly, I'm surprised that you don't
> know this already. How exactly did you perform updates in the last 3 months?

As you see I do know how *I* do it, what I don't know is how *YOU* do
it.  Would you please tell me?

To conclude, I'm perfectly all right with your comments and questions
above, however from your tone I'm getting the feeling that are
irritated with me.  This hasn't been my intention at all.  What I want
is for you to share what you've learnt so that I can learn from it
too, and ideally this would also lead to improvements in the tools and
processes we use to work with HABS.

/M

-- 
Magnus Therning                      OpenPGP: 0xAB4DFBA4
email: magnus at therning.org   jabber: magnus at therning.org
twitter: magthe               http://therning.org/magnus