new Library Infrastructure spec.

[Isaac Jones]

Hi Arie,

Arie Peterson <ariep at xs4all.nl> writes:

> At 18:12 2004-06-03 -0400, Isaac Jones <ijones at syntaxpolice.org> wrote:
>>We're trying to give the end users (and tools that may be layered on
>>top of the packaging system) a consistent interface to packages whose
>>needs and underlying implementations are very different.  (Think of
>>the different kinds of needs between HUnit and wxHaskell).
>>
>>Now Distribution.Simple is a particular implementation of this
>>interface which is meant for very simple libraries and tools.
>>
>>What you propose to provide is a secure infrastructure for building
>>and installing very simple libraries and tools, but not providing a
>>common interface across diverse libraries and tools.  I feel that this
>>is a completely different goal.  (I also think that if this is what's
>>desired, the user is better off using a packaging system that has more
>>guarantees, and not installing code from untrusted sources at all (see
>>my work in "apt-secure" btw) but that's neither here nor there.)
>
> I do not understand why a strictly defined data structure would not be
> a common interface, and why a (for ignorant windows users like me, at
> least) obscure and opaque makefile would.

I'm afraid you must misunderstand something.  A makefile is not the
interface.  The interface is the Setup script and the arguments to it
(install, build, etc).

> Do you mean by 'interface' the way in which a user invokes the
> installation? I would like to be able to choose whether I type "make
> install" myself, 

I don't understand; the "choice" of how to install the system is not
made by you (Joe, Bob, or Sam), the installer, but rather by the
author (Angela or Marcus).

> let a linux package manager handle that for me, or perhaps use some
> customised homemade install manager to give me exactly the
> information and options I want.

This doesn't replace RPM or Debian-type package management; it
facilitates it (for Ronald, Donald, and Willie) .  For instance, the
"common debian build system" can be designed to wrap the Setup
interface; making it very easy to make Debian packages for Haskell
libraries and tools.  In particular, all a packager has to do for a
package that conforms to the interface, is write a "control" file that
says "build this like a standard haskell package".

I already implemented a prototype of this, and it works nicely.

OTOH, for those of you who aren't lucky enough to have a nice package
manager on your operating system, the Setup script will provide a
consistent interface for installing packages, and in the future
perhaps for wrapped tools for downloading and installing packages from
a remote database, such as CPAN.

> Alex' idea of an installation data structure suits that wish
> perfectly. As mentioned before, it would be particularly nice to
> know what files/directories are to be modified or added.

Package managers such as dpkg typically handle this kind of thing.  If
you use one of those, you'll still have this ability for packages
installed via dpkg.

However, providing such a facility is not possible for packages by
Marcus.

This kind of security is properly handled on the operating system
level.  If you are downloading random code from the web and installing
it outside your package manager, haskell packages are the least of
your problems.

>>Distribution.Simple provides some mechanisms for making Angela's life
>>easier, especially with very simple libraries. Now say Angela has a
>>library that uses Distribution.Simple, but she's altered it to utilize
>>the Happy preprocessor, and (let's say) Distribution.Simple doesn't
>>know anything about Happy.  Now Angela can add a "system" call to her
>>Setup file to preprocess what she needs to, and continue using the
>>rest of the Distribution.Simple mechanism she was using before.
>>
>>In the StaticSetup scheme, Angela is out of luck.  She can't change
>>the behavior of StaticSetup to do the right thing, so she has to get
>>rid of her build system altogether and start from scratch with
>>makefiles or something.  This is what we want to avoid.
>
> If the preprocessor is to be executed by the user, 

Which user are you referring to?  We gave them all names :)

> then this is just a
> dependency and can be listed as such in the install structure.
> If the preprocessor is to be executed by Angela, then she does so
> happily (sorry ;-)) and the user need not know that a preprocessor was
> used.

I'm confused by this.  Maybe if you told me which user you're
referring to I could explain it.

> Let me suggest a different approach.
>
> We could define a Setup/Installation monad, together with some some
> primitives (display a message, ask for user input, move files, check
> dependencies, request installation of other packages, etc.). One
> special primitive might be 'IO', containing an arbitrary IO operation,
> to accommodate installation actions that cannot be defined in terms of
> the given primitives.
>
> A package description/installation might consist of a haskell data
> structure, holding some basic properties of the package and a Setup ()
> structure.

If the installer system, provided by Angela, can perform arbitrary IO
actions, then you have lost the property of knowing which files it
installs.

> This package structure may be distributed:
> 1. as haskell source code alone, to allow users to install the package
> with their own customised installer or a default ghc installer;

How does the user customize an installer for darcs?

> 2. in source form with an installer that the user may or may not use;
> 3. compiled by the author with her installer to produce a binary
> executable installer.

Our system covers 2 and 3.

> Notice that an 'installer' in this context is a haskell program,
> typically run with hugs or ghci (unless dynamic linking will be widely
> supported), that merely has a Setup () as a parameter and interprets

I'm confused by who is providing this installer, and how this Setup
structure gets provided to an executable.

> this structure on the fly, in a way that the author of the installer
> thinks best: install the package in the right directory, do or do not
> download necessary libraries automatically, do or do not ask the user
> to authorise the IO parts, 

Why ask the user if they want to authorize the IO parts?  They will
have to read the code to find out what the IO actions do.

> register the program with the compiler of
> choice, etc.. The installer might get settings from a conf file or
> bother the user with every detail - that is up to the author of the
> installer.
>
>
> I feel somewhat awkward to suggest this wild idea, because the
> proposal seems well-thought out and quite mature.
> So please convince me that this approach is useless :-).

I'm afraid I do not completely understand your approach.  Most
especially, I'm confused about what it buys you that our approach does
not.  I do suspect that we're losing support for Marcus, which is not
acceptable.

Further, I think that once you completely understand our approach,
you'll like it.  Please provide suggestions for the proposal for bits
which are unclear.


peace,

  isaac