FW: Library infrastructure

[Simon Peyton-Jones]

Gentle Haskellers

Haskell has now got a decent, hierarchical module system, supported by
GHC, Hugs and NHC, and the library structure is now quite well
populated.  For example, here are the "base" libraries shipped with GHC
(similar libraries come with Hugs and Nhc):
	http://www.haskell.org/ghc/docs/latest/html/base/index.html

Languages flourish when libraries are plentiful, reliable, and well
documented.  In an open-source community like the Haskell community,
plentiful libraries depend on contributions from the community. =20

	The difficulty is that it is currently not obvious what Joe
Programmer=20
	should do to contribute a library to the community.

	I think this difficulty is a major obstacle to Haskell's
progress,=20
	perhaps even the largest single obstacle today.  =20

One route to getting a library widely distributed is to get it into the
"libraries" source tree used by GHC, Hugs, and NHC.  That has the merit
that the library will be distributed with each of these compilers.  But
it's too much of a "cathedral" approach.  There is no way to distribute
libraries between releases, and it makes the distributions ever larger
and more complex for the GHC/Hugs/Nhc teams to build. =20

To complement this, there ought to be an easy "bazaar" way for Joe
Programmer to implement a library, build distributions, put them up on
his web site for others to download, and add a link to some central
library index so that people can find it.  Sadly this is not an entirely
simple task, because (a) the library should work on all Haskell
implementations, and (b) should be packaged for binary installation on a
variety of platforms (Windows, Linux, OpenBSD, Mac etc).   Indeed, since
GHC, Hugs and Nhc have different packaging mechanisms, there's a kind of
(a)*(b) product involved.   The bottom line is that today the obstacles
are high enough to discourage all but the most motivated.

What is needed is some kind of software infrastructure to automate the
common tasks, and to give clear instructions about the non-automatable
parts.  This infrastructure should not be part of the (complicated) GHC
(or Hugs, or Nhc) build systems; indeed, while it will certainly contain
implementation-specific parts, its structure should be implementation
independent.   Simon Marlow emitted a message a while ago about this,
but discussion petered out:
=20
http://www.haskell.org/pipermail/glasgow-haskell-users/2002-November/thr
ead.html#4322

The lack of this library-building infrastructure is a serious problem.
Solving it, or even partially solving it, would be a huge step forward.
So this message is really to try to kick-start the process again.  Is
anyone willing to invest effort in making this happen? =20

It's probably a task that is too big for one person.  I think what we
need is

    A library-infrastructure Czar, who will=20
	a) do the core design and implementation
	b) coordinate the whole thing to make sure progress happens

    A bunch of library-infrastructure Supporters, who are willing
	a) to contribute thoughtfully to the design
	b) to implement pieces of it, for example pieces specific to one
implementation
		(e.g. GHC) or one target architecture (e.g. PowerPC)


Perhaps the first thing to do is to assemble list of people who would be
willing to contribute as supporters (in the above sense), hoping that a
Czar may then emerge by popular acclaim.  Would you like to reply,
perhaps to the 'libraries@haskell.org' list, so indicate interest.

	I really think this is important.  If we do not collectively
evolve a way
	to develop libraries in "bazaar" mode, I think Haskell will at
best develop
	more slowly, and at worst wither away.

Simon

PS1:  This is all distinct from the question of library *content* ---
that is, what the module name, discussion about the signature of the
exported types and functions are --- which does seem be handled OK by
the 'libraries' list.  This message concerns library-building
*infrastructure*.

PS2: Another entirely-orthogonal question is where library source code
*lives*.  One possibility is to use the same CVS repository that
GHC/Hugs/Nhc use for libraries, albeit in a separate subtree.  Perhaps
fptools/user-libs/joe-bloggs/*.  Another possibility is SourceForge.
Finding backed-up CVS space for source code is not a problem.