new Library Infrastructure spec.

[Isaac Jones]

The packaging system (or "Cabal"[1] as it's called now) is just one
part of an over-all infrastructure that we would like to build some
day.  We think that building things in layers is important.  The
bottom layer should be the compile / install infrastructure, so that
tools which help with building and installing can be layered on top of
that.

The tools should do their job, and do it well.  The compiler compiles,
the build / install system installs, and you can layer a tool on top
of that which tracks package databases.

For discussion of this, you could read this brief section of the
previous proposal (which if you haven't read might provide some useful
insights), and some of the old use cases.

http://www.haskell.org/libraryInfrastructure/proposal-1.0/x396.html

It is very important to me to support tools by Marcus.  His packages
are actually the most difficult to manage, and the most confusing for
an end user.  Supporting just tools by Angela would in fact be much
easier (except perhaps for implementing things directly in the
compilers, as you propose).  Fortunately, I think we can support
Marcus without making Angela's job harder.

If we can step back for a moment, I think you're saying that
supporting Marcus isn't that important, or that it's too hard.  Maybe
we could break up the argument along the lines of:

1) Why we should / shouldn't support Marcus

2) What would / could a system that just supports Angela look like

3) What would a system that supports tools by both Angela and Marcus
   look like (this proposal is our answer to that).

I'm fairly comfortable that our system is appropriate to answer 3.
Perhaps I haven't given enough justification for 1?  I haven't thought
about 2 very much because in my experience, that's the least of my
Haskell-related problems, and 3 subsumes 2.  Maybe you could give more
explicit argument for why 2 is so compelling as to exclude tools by
Marcus.

(more below...)

"S. Alexander Jacobson" <haskell at alexjacobson.com> writes:

> On Mon, 7 Jun 2004, Isaac Jones wrote:
>> > Any reason we can't simplify Joe's interface to:
>> >
>> >    ghc  --install http://url/of/some.pkg
>> >
>> >    --or--
>> >
>> >    hugs --install /path/to/some.pkg
>>
>> This is quite explicitly something we're trying to support.  The
>> original document, and my talk at the Haskell Implementor's Meeting,
>> had some exampled of "layered tools" for the system, including exactly
>> this use.  I should probably re-add that to this version of the
>> proposal.
>
> This interface should be _fundamental_. JoeBob*,
> and Sam are indifferent to how it is implemented.
> It should just work if the compiler/interpreter is
> installed correctly.

I would be open to discussing this further if we could get all the
compiler authors to agree to implement it.  Even if they did, it could
still be done using Setup as the interface.  This is an orthogonal
issue.  (And as an aside, I'm not sure if it makes sense to invoke a
compiler to install stuff.)

> [...sample shell script implementations elided
> because implementation is irrelevant to this
> discussion... 

They were there to demonstrate how they can be easily implemented once
we fix the Setup interface.  This demonstrates the power and
flexibility of this interface.  I'm not sure why you think it's
irrelevant.

> question: Why are the sample implementations not themselves in
> Haskell!?...]

It was just an example.  Could just as well be implemented in haskell.
I know that there's a haskell wrapper for libcurl (which would take
the place of wget).  I don't know of a haskell unzip library, and I
don't know if libcurl is there for windows.

(snip)
>> > The open question is what this setup specification
>> > should look like.  There is an obvious tradeoff
>> > between the interests of Joe and Angela on the one
>> > hand and Marcus on the other.
>>
>> I think our scheme satisfies Joe, Angela, and Marcus.  All Joe knows
>> is that he says "haskell-install".
>
> At least in the proposal so far, someone has to
> deal with "runhugs" or whatever.  --install should
> be standard for every compiler/interpreter just
> like --package.

Haskell-install deals with the execution of the wrapper script.  It
does so in a platform-dependent way, and can be made to understand
users' compiler preferences or what-have-you.

>> All Angela knows is that she fills out the (declarative) fields
>> required by Distribution.Simple.  She doesn't know or care how Joe's
>> operating system implements wget.
>
> Or even IF Joe's os implements wget.  Why not
> implement in Haskell to avoid this issue?

That's perfectly fine.  The shell script was just an example.

>> Angela does have this.  She can use Distribution.Simple.  She's not
>> doing arbitrary or obscure IO.  The example we give is very
>> declarative for her.
>
> Sorry, where is the example.  1.2 does not include
> telling the system that Angela.Internals is
> private to Data.Set and Data.Bag.  3.4 is empty
> as is 5.3.
>
> My concern here is in particular that the
> (declarative) fields are not yet fleshed out even
> though they are the most important part of this
> sort of proposal!

The interface is the most important part, and in our proposal, this is
not the interface, though you may assert that it should be.

We actually have a really good idea of what the fields are, but not
what the format is.  Fortunately, once we've defined the interface, we
can put off being precise about the fields. If you get a chance to
take a look at the code, the package description looks like this right
now:

data PackageDescription
    =  PackageDescription {
	-- the following are required by all packages:
	package        :: PackageIdentifier,
        licenese       :: License,
        copyright      :: String,
        maintainer     :: String,
        stability      :: String,

	-- the following are required by the simple build infrastructure only:
        buildDepends   :: [ Dependency ],
        allModules     :: [ String ],
        mainModules    :: [ String ],
        cSources       :: [ FilePath ],
	exposedModules :: [ String ],
        extensions     :: [ Extension ],
        extraLibs      :: [ String ],
        includeDirs    :: [ FilePath ],
        includes       :: [ FilePath ],
        options        :: [ (CompilerFlavor, [String]) ]
    }

>> Autoconf and friends implement this.  We provide an interface which
>> wraps both autoconf and Distribution.Simple.
>> Sam has been perfectly happy for years using "./configure;make;make
>> install".
>
> So, why are you making Sam do something different
> for Haskell?
>
>> OTOH, Marcus needs something more flexible.
>
> Is there some substantive problem with
> autoconf/make with respect to Haskell such that it
> needs to be wrapped with Setup.lhs?

No.  It's a matter of providing a common interface to users and tools.

> If I have a package that is part C, part Python, and part Haskell,
> why is it reasonable to assume the outer-layer is necessarily
> Haskell as opposed to e.g. Python's dist-utils?

Packaging depends on your target audience.  If the author of such a
package doesn't want to implement the Setup structure, there's nothing
forcing him to.  His users just won't be able to use haskell-install.
If they're mostly Haskell users, then this will annoy them.  If not,
then they won't care.

> Or does Marcus need to strategize whether the outer layer is
> dist-utils or HPS and then have one call the other?

If he cared that much, he could implement both.

> * I don't believe Joe and Bob are different people
> so I am consolidating them as JoeBob.  In
> particular I don't believe that Joe user should
> have to get someone else's help to do a simple
> install.

Joe doesn't install new packages, nor does he ask for packages to be
installed.  He just wants to invoke the haskell compiler.  Picture a
student in a class where all of the libraries and such are already set
up for them.


peace,

  isaac

[1] Cabal: Common Architecture for Building Applications and Libraries