Recommendation for the procedure to add platform packages

[Duncan Coutts]

On Fri, 2009-08-21 at 02:23 +0100, Ian Lynagh wrote:
> On Thu, Aug 20, 2009 at 03:04:20AM +0100, Duncan Coutts wrote:
> > 
> > There is also an example proposal:
> > 
> > http://trac.haskell.org/haskell-platform/wiki/Proposals/example
> > 
> > So please send in your comments and of course feel free to ask questions
> 
> I read the example first, so I'll give my comments on it first:

The policy is significantly less prescriptive than the example might
lead you to believe.

The essential requirement is given as:

        The proposal forms the nub of the argument for why the package
        should be included in the Haskell Platform. It should contain or
        link to enough information so that reviewers can make an
        informed decision on whether the package should be accepted.

With further specific details given in the "Proposal content" section:
http://trac.haskell.org/haskell-platform/wiki/AddingPackages#Proposalcontent


I should also note that I'm happy to improve my tar proposal to pick up
some of the points you make. In addition to being an example proposal I
intend to submit it as a real proposal, so improvements benefit both the
real proposal and the example.

> My first impression is that there is an awful lot of text, and that it's
> mostly in paragraph form (i.e. I need to actually read it, rather than
> the key points being highlighted). I think this will reduce the level of
> review that proposals get.

Below you suggest that instead the proposal should link to existing
documentation (and that such documentation be required to exist). I
presume the documentation linked to would have the same amount of text.
Assuming that's what you mean, I'm not sure what you're comparing to
that has less text that we might expect would give us an increased level
of review.

>     This is a proposal for the 'tar' package to be included in the next
>     major release of the Haskell platform.
> 
> Should specify version number.

I expect that usually the exact number is not especially relevant. The
exact released version now is not necessarily the same as the final one
that gets included in the case that reviewers make suggestions.

If there's more than one active branch then yes it's relevant so
reviewers know which one to look at. Otherwise reviewers can reasonably
assume the version on the hackage page.

The guidance notes also mention that if reviewers are expected to have
to look at a development version, that the repo should be given in the
proposal.

>     Review comments should be sent to the libraries mailing list by
>     ${deadline - 4 weeks}
> 
> This sounds like it is the wrong way round to me. The deadline for
> review comments should be "${now + 2 weeks}" (for some value of 2, and
> larger values for complicated/large proposals). If you make the proposal
> early, then it can be agreed earlier. If you make it too late for the
> discussion and consensus to happen by the deadline then it can wait
> until the next release.

Suggesting that comments be made by a time relative to the proposal time
rather than the final deadline would also be consistent with the policy.
The constraints are that people be given a reasonable amount of time to
review things and that discussions and issues are resolved before the
final deadline.

I'd be happy to change the guidance to suggest now + time rather than
final deadline - time.

> (that's pretty much how library submission proposals work).

Sure. Though I think we need to clarify how much time is reasonable for
review rather than letting the proposers pick whatever they like.

>     Abstract
> 
> I think this should be replaced with a link to the hackage page, which
> shows the abstract from the Cabal file. You might argue that having all
> the info in one place is convenient for reviewers, but
> * I am unconvinced that it is much easier
> * Packages that are proposed are probably widely used anyway, so this is
>   just noise to a significant proportion of people
> * by only linking to it you remove the temptation to alter the text in
>   the proposal but not apply the improvments to the Cabal file.

The policy allows the info to be inline in the proposal or linked to.
It's up to the proposal author:

        Where appropriate, existing sources of information about the
        package may be copied or linked to.

>     Rationale
> 
> I don't think the actual rationale part of this section is useful. It
> mostly says "The foo library is useful because some people want to foo".
> If during the discussion we find that the rationale is unclear then a
> rationale could be added to the proposal, but mostly I expect it will be
> self-evident (and if not then the abstract in the Cabal file probably
> needs some work).

It's intended to cover these proposal requirements:

        The proposal forms the nub of the argument for why the package
        should be included in the Haskell Platform.

        It should explain the gap in the existing set of platform
        packages that the new package addresses. Alternatively if the
        package would duplicate or replace any existing platform
        packages then the reasons why the new package is better should
        be clearly explained. In this case the proposal should be tied
        to another proposal to deprecate the existing package(s) (or
        parts thereof) and thought should be given to how the transition
        should be managed. [note-7.2]

The guidance notes say:

        The rationale is the core of your argument as to why this
        package should be added to the platform. Your task here is to
        convince people. What makes the package useful? What are the use
        cases? Being able to point to existing uses is always good.
        Similarly, existing comments or reviews from users can be
        helpful. Old release announcements can be a good source of
        inspiration here.
        
        If your package is a replacement for an existing package (or
        part of one) then you'll want to convince people why the new one
        is an improvement.

I agree that what I've written for the rationale for the tar proposal is
of somewhat dubious utility. In fact personally I don't like the term
"rationale" for the section.

> The list of users is useful (less convinced by the list of packages you
> think /ought/ to use it), but a bullet-pointed list would be much easier
> to read.

Well I'm convinced darcs ought to use it ;-). I'm citing it there as an
example of an alternative to using the tar package, to use an external
tar program (and the various disadvantages that has).

In general I don't think it's an unreasonable argument to point out
cases where existing packages could be improved by making use of the new
package rather than using some existing home-grown solution. Of course
this is all a judgement for the proposal author for what they think
makes a convincing argument (and for the reviewers to decide if they're
convinced). I'll keep it in mind that you don't find it convincing :-).

>     Introduction to the API
> 
> What I said about the abstract applies here, only moreso. If an intro to
> the API is useful, then one should already exist.

It's explicitly ok to link to one where one already exists. The policy
and guidance notes say as much.

> People should not be given the opportunity to /write/ an introduction
> to the API here; it should already be in the haddock docs on hackage,
> or on the project page on community, or in some similar location.

If we add that as a package requirement then it would be reasonable to
change the advice to say simply to link to it. Currently the only doc
requirement we've put in is:

        Library packages should have Haddock API documentation for all
        exported functions and types.

I expect once we've agreed the procedure that we'll get onto more
discussing more detailed requirements on things like documentation.

In the tar example I had already made an introductory section to the API
in the haddock docs. However on trying to present it again in the
proposal (by doing a fair bit of copy and pasting) I think I made some
improvements to how and the order in which concepts are presented.
Writing with a particular audience in mind can help a lot.

> Therefore this too should just be a link. Again, this will remove the
> temptation to update the proposal but not the real docs following the
> discussion. It will also mean people don't waste time reformatting
> docs in wiki markup.

The guidance notes say:

        Of course, if you end up making something new for the proposal
        then it'd be great to keep it somewhere so that new users can
        benefit from it too. The obvious place is in the Haddock docs,
        or a project website.

>     Design decisions
> 
> Ditto.

The same points apply. If we want to require that packages must have
this kind of documentation then it ought to be spelled out explicitly as
a package requirement. Currently it's just part of the proposal because
it's a less onerous requirement.

>     Open issues
> 
> This is the interesting bit of the proposal, IMO. What I want to see for
> a proposal is just something like:

That's a quite different kind of thing from what the open issues section
is intended for. The policy says:

        Open issues and objections raised by reviewers should be tracked
        on the proposal wiki under a separate section. [note-3.2]

and

        An explicit checklist of the package requirements below is not
        required. The proposal should state however that all the
        requirements are met, or for any requirements that are not met,
        a reason why they are not met. [note-7.6]

What you're asking for is a checklist. That idea was particularly
controversial within the steering committee and it was not adopted.
Personally I was weakly in favour if a checklist but the concerns were
that it restricted the arguments and explanations that the proposal
author could make. The consensus was for a much more free-form proposal
rather than forcing things into a checklist.

> and the goal of the discussion would then be to get a consensus on
> whether the yellow sections are severe enough to not accept the package
> (and also to give people a chance to disagree that some of the green
> entries really are green, e.g. "Is useful?" is subjective).

I think that would come under the complaint that that is too narrow for
the argument for why the package should be included. Perhaps other
members of the steering committee would like to comment on this point.

On the other hand if it's not part of the proposal then I think you'd
get a better reception. I know that the release team will want to use
its own status board for the aspects of packages that particularly
affect the release team (like building on various arches).

> I've mentioned this link on IRC before, but for those who haven't seen
> it, this is partly inspired by:
>     http://release.debian.org/etch/arch_qualify.html
> (which shows at a glance which arches are acceptable to be in a Debian
> release, and where the issues and potential issues are. The link in the
> first row gives more detail on some of the points).

My concern if this is the main focus of discussions is that it covers
the "does it work" aspect reasonably well, it doesn't cover the "is it
any good? how could it be better?" aspects.

> >
> > http://trac.haskell.org/haskell-platform/wiki/AddingPackages
> >
> 
> I've only skimmed this, due to the length, but my impression is that it
> is too heavy-weight. e.g.
>     http://www.haskell.org/haskellwiki/Library_submissions
> has this to say about consensus:
> 
>     If someone has done all this, they are entitled to expect feedback;
>     silence during the discussion period can be interpreted as consent. 
> 
>     If consensus was achieved, [...]
> 
> and doesn't seem to have had problems with getting consensus, or
> agreeing whether we have got it.

That's the main reason we agreed to go with a consensus method. The
policy states:

        The standard of consensus required is the same as that defined
        for the existing library submissions process. [note-5.4]

Our early draft simply said that and nothing else. Several members of
the steering committee and others who read drafts were unsure or
unconvinced that consensus would be a usable method. We agreed to try it
and see if it works. In addition we added the option to use this
consensus protocol.

> The "Achieving consensus" section of
> AddingPackages is 947 words according to wc (that's about double the
> length of the entire Library_submissions page).

You will notice that in the simple case there is essentially no
formality at all:

        The first stage is presenting the proposal followed by open
        group discussion. This stage lasts long enough to give everyone
        the opportunity to send in their review comments. There follows
        a call for consensus.

This is essentially the same as the libraries process (including the bit
about expecting feedback and silence is consent). We only need to go
further when there are remaining disagreements.

So I do not agree that it is too heavy-weight. It starts out very light
and gets more structured as the severity of the disagreement increases
and as the final deadline approaches.

> Also, the fact a credits section was deemed necessary suggests to me
> that too much work is involved.

It was agreed because it's a cheap and easy way to promote the work of
review which is often undervalued. We also think that serious review is
a great way to improve package quality and entry to the platform is an
appropriate moment to do it.

There is clearly a trade off between:
      * work for package proposers / maintainers
      * work for reviewers
      * quality of package review and quality of decisions

My impression so far is that opinions are reasonably evenly split on all
three. To me, that indicates we're roughly in the right place.

What are you suggesting specifically that would reduce the amount of
work for reviewers? Would those things increase work for the proposal
authors / package maintainers? Would those things increase or decrease
the level of depth/quality of review and decisions?

Of course if there are any suggestions for straightforward wins in this
trade-off then we should consider them seriously.

Duncan