Haskell library infrastructure coordinator

[Isaac Jones]

Greetings all.

> Simon Marlow and I would like to propose Isaac Jones for this
> coordination role.

Thank you :) I'll wait to hear if any objections come up, but in the
mean time...

"Simon Peyton-Jones" <simonpj@microsoft.com> writes:

> Lots of suggestions emerged, and lots of people have knowledge and
> willingness to help.  

There are already some comments from others that I'll reply to
separately.

> What we need, though, is someone to coordinate this group.  It's
> difficult to reach technical agreement in a distributed group without
> one person who is willing to
> 	* Make proposals

I have started on a proposal that I put on the wiki and I'll paste in
this email [1].  Perhaps before proposals are really addressed, we
should come up with a set of requirements.  Does anyone want to
propose a set of requirements?

For what its worth, my initial proposal, to get the ball rolling, is
here: http://www.haskell.org/hawiki/LibraryInfrastructure

> 	* Moderate discussion

I feel that discussion should happen on this mailing list and perhaps
a little on the wiki.

> 	* Document the consensus

I plan to document the consensus on the Wiki page mentioned above.

> 	* Restrain ambition: something modest that works is better than
> 		something aspirational that doesn't

Its already been said that my proposal is too ambitious, but one of
its strengths is that it can be implemented incrementally and in
layers so that each layer is useful on its own, but is specifically
designed with "hooks" to be useful to the upper layers.

> A huge thank-you to Isaac.  Haskell only works because people contribute
> to it.

You're very welcome.  I think this issue is indeed an extremely
important one and I am excited to be a part of the solution.

peace,

isaac

[1] Much of the discussion on this issue has focused on ideas to
retrofit "pickOneFrom [Python, Debian, FreeBSD, fptools, hmake, CPAN]"
to Haskell.  This may not, however, be appropriate.  I propose a
combination of appropriate concepts from each system, reimplemented
for Haskell.  As such, this proposal is somewhat more ambitious than
the other suggestions, but I feel that this is warranted.

This proposal is based on a layered tools approach.  The layers can be
built and used incrementally, but it is important to keep the higher
layers in mind when implementing the lower layers. 

== Building ==

 * Each sufficiently complex library should have a robust build
system, perhaps based on the fptools makefiles.  (Alastair Reid seems
to be working on cleaning this up) A program to generate a skeleton
can be provided.

 * Standard library search paths might be necessary, and standard
flags would be useful.  hmake could go some of the way (maybe it
already does?) in hiding different compiler behavior where possible. 

== DistUtils ==

 * The make system should be wrapped with a python-style distutils
system which provides a ./setup.hs with standard targets so that every
library gets built and installed the same way (this standardization
will help to build tools around the infrastructure, as Debian
does). Simpler libraries that don't need something as complex as
fptools can use just this script.  Projects that don't want to use
fptools system can wrap their own system with this script.  Python
provides some standard useful classes that can be imported to help
with building and installation, and Haskell can do the same.  This
would require some support like a standard /usr/bin/haskell
interpreter that works like runhugs but invokes the user's default
compiler.  hmake could probably be altered to serve this purpose
considering that 'hi' gets us part way there.  The tool that Henrik
Nilsson mentioned might help a lot too.

 * Some meta information could be kept in a setup.cfg file or in a
subdirectory.  This would be similar to the way Debian is able to
standardize the build process so well.  This setup file might have
build dependencies, a package name, version number, etc.  More about
the meta information later.

 * hmake already keeps a list of installed compilers and a default
compiler.  The setup.hs script could have default targets like
"./setup.hs build all" or "./setup.hs build default" or "./setup.hs
build ghc5" which would build for the given compiler.

Then "./setup.hs install" would install and perhaps register each
package that was compiled on this machine on a per-compiler
basis. When a new compiler is installed, it'll go about recompiling
registered libraries where necessary so that end users don't get
"permission denied" errors when building their own software that
depends on libraries built with older versions.  When a compiler is
removed, it could remove the binaries for libraries associated with
that version.  This might be the hardest point: getting over binary
incompatibility.

The registering is for software compiled on the end-user's
machine. This is specifically for systems like Solaris or for
libraries dropped into /usr/local, ie packages whose source is
available but who are outside a packaging system which solves this
problem some other way. 

== Packaging ==

 * Now how would this situation interact with systems like FreeBSD and
Debian?  I envision a tool for Debian which will look at the meta
information in the setup.cfg file and the installed compilers, and
build a skeleton "debian" directory which would help a packager adhere
to a Haskell policy so packages are all consistent; man pages are put
in the right section, libraries are put in the right places, etc. This
could also help in the creation of separate binary packages (from one
source package) for each compiler.  I don't know much about FreeBSD's
ports system, but I imagine that a similar thing would be a bit easier
for a source distribution.

== Distribution ==

 * We might also want a CPAN-like repository for authors to put
Haskell related libraries which use this build infrastructure so that
dependencies can be easily downloaded, etc.  This is a pretty separate
issue, but it would be really great to have.  Sourceforge might be
good for now, but something like what Debian has would be great.