Proposal: Change to library proposal process

[Simon Marlow]

It's interesting to look at why we introduced the process in the first 
place:

http://www.haskell.org/pipermail/libraries/2006-October/006005.html

  "The idea is to reduce the number of 'bikeshed' discussions
   we're having, ensure that submissions are not dropped, and
   generally improve productivity."

that is, pretty much all the things that people are complaining that the 
process is not doing.  We had good intentions, but I guess it didn't 
work.  It didn't even ensure that submissions are not dropped, because 
we did end up with plenty of abandoned proposals.

Some of us have been discussing this at length on IRC over the past 
couple of days.  With it being a new year this is great time to take a 
critical look at our processes, and I agree with Simon that we should 
take this quite seriously.  I'd like to thank Greg and Johan for 
sticking around to discuss this rather than giving up.

The underlying problem they've raised is that this community has become 
a little too afraid of change.  Nobody feels empowered to do anything 
without first consulting "the community", and that typically ends up 
nowhere except for very uncontentious issues.  However, historically all 
the progress has been made by individuals or small groups Just Doing It. 
  Hackage has enabled the community to share the fruits of each other's 
productivity on non-core libraries, but very little movement is 
happening at the core.

The slow rate of change is to some extent a reaction to a period of 
instability we in the past (around GHC 6.8) when there was a lot of 
upheaval and complaints from users that their code kept breaking with 
new GHC releases.  We took some positive steps to help with this (the 
base-3 compatibility package, wider use of deprecation, and generally 
shying away from breaking changes).  There has always been a tension 
between improving APIs and stability, and currently we're skewed towards 
stability, at least for these core APIs.

Compulsory review of API changes is a good thing - we get lots of eyes 
on the changes.  However, the process would be more appropriate if we 
were basically happy with the APIs we have.  On the contrary, we have 
plenty of large-scale changes that need making in base and other places, 
and forcing all changes through the library process is making it hard to 
get these cleanups done.  Another thing making it hard is the lack of 
people actually doing it, but it's difficult to tell whether that's due 
to the obstacles we've put in the way, or whether there really aren't 
people with the time and inclination.  We'll only find out by actually 
removing the obstacles.

The process is not applied consistently either - there are libraries 
like bytestring which are not under the process but are arguably just as 
"core" as containers.

I'm coming around to the view that this small-scale API change review 
isn't getting us where we want to go.  To really improve APIs we should 
have a group of people looking at the whole, and making strategic 
decisions.  Let's figure out where we're going and how to get there, 
rather than making a series of tiny incremental steps (slowly).  Let's 
really make FilePath an ADT, make binary Handles a different type, move 
more of base out into separate packages, remove the Show superclass of 
Num, overhaul the containers API, finish creating the Exception 
hierarchy, etc. etc.  We've had a period of stability, maybe now it's 
time for change, and lots of it.

The changes being proposed by Greg and Johan, as I understand it, would 
amount to the following.  I'm willing to give it a try; we can always go 
back if it doesn't work out.

  - maintainers are empowered to make API changes.

  - no formal review process for API changes, although there is an
    expectation that non-trivial changes would still be discussed on
    the list beforehand.

  - we would still have a clearly documented path for contributions
    and proposals

  - commit messages are copied to the list, and should
    detail API changes sufficiently that the community
    can keep an eye on what is happening and comment.

  - we get more maintainers (presumably they'll just appear, or
    something? :-)

Does anyone violently disagree?  Suggestions for improvements?

Cheers,
	Simon