Extending the dependency syntax

[Simon Marlow]

On 11 August 2005 04:38, Isaac Jones wrote:

> "Simon Marlow" <simonmar at microsoft.com> writes:
> 
> (snip)
>> build-depends:
>>           --enable-ghc? (
>>               ghc>=6.4  ? [ghc64] |
>>               ghc>=5.04 ? [ghc-old])
>>         | --enable-hugs? [hugs]
>>         | true ? true,  -- other compilers are allowed
>>         --enable-debug? (HUnit-1.0, [debug]) | true? [release]
>>         --enable-gnome? (
>>                  libglade >= 2,
>>                  gtksourceview >= 0.6,
>>                  gconf >= 2, [gnome] ),
>>         --enable-mozilla? ( mozilla >= 1.4, [mozilla] ),
>>         --enable-doc? ( haddock >= 0.6 )
> 
> What happens now with the issue that Duncan mentioned, where the built
> system picks up configuration information from the build / configure
> environment that's different from the user's environment?  It could
> either pick up too much or too little.  For "too little" we could use
> the --enable flags to override the detection mechanism, and maybe a
> --disable-all-detection mechanism similar to the -fhide-all-packages
> mechanism would be useful.  One problem is that --enable-foo doesn't
> necessarily exist for optional dependencies.

The example I wrote above is deterministic: it doesn't pick up
dependencies from the environment and change its configuration
automatically (this isn't accidental).  I'm going with the
derministic-dependencies-by-default approach, which seems better.

It's possible to write non-deterministic dependencies with this scheme.
Perhaps that's a problem.

A few messages back I mentioned that we might have an --auto-enable
switch to get back the handy non-deterministic behaviour.  I can't
immediately see how to make it work (eg. we don't want --enable-debug
enabled automatically if HUnit is available).  But I don't consider the
lack of --auto-enable to be a big deal - good OS packaging systems like
Gentoo provide this anway.

> Another point: The dependency syntax you show mixes packages and
> pseudo-packages like GHC, alex, etc... I don't think we should use the
> same syntax / field for this, since in reality, they have separate
> namespaces.  In fact you might be insane enough to want to talk about
> them separately: maybe you have the haxml library installed, but not
> the haxml toolchain... maybe you want the ghc executable, but not the
> ghc package...

Ok, but separating these dependencies will mean that you can't combine
them.  eg. you can say "if I have GHC 6.4 and wxHaskell then ... else
...".  And separating them seems to make the system more complicated
rather than less... but I don't feel that strongly.

> We used to have a distinction between depends and build-depends, but
> we threw that away, I think because we weren't using the
> non-build-depends stuff at all... I note that having separate depends
> vs build-depends fields doesn't actually fix the above problems.  BTW,
> are you suggesting that we change the meaning of depends and really
> adding a build-depends field?

I'm just suggesting extending the syntax of build-depends in a
backwards-compatible way.

> There are at least four kinds of simple dependencies, not taking into
> account configurations:
> 
> 1) build-depending on packages - which is what depends: means now.
>    This is always a build-time dependency.
> 
> 2) build-depending on executables like GHC - that the simple build
>    infrastructure knows about / needs.  It would also be nice if we
>    had a more sensible way of adding --with-foo= flags,
>    extra-foo-opts, and configure-time detection for stuff in this
>    category.

I put some thought into the --with-foo thing too, and Dimitry's
suggestion, but again I think we should deal with one thing at a time.

> 3) run-time depending on executables like gpg, gnome, etc.  These are
>    likely to be executables that Simple has never heard of before.
> 
> 4) build-depending on stuff that the simple build infrastructure
>    doesn't know about.  This may not be very common, but folks not
>    using Distribution.Simple may want to use this field to express
>    dependencies too, so it would be nice if we could express
>    build-time dependencies on non-package tools which we don't already
>    know about.
> 
> In reality, there are even more kinds :(
> 
> The original request, I believe, was for better support of
> tool-depends mainly 2, but perhaps 3 and 4 are also important?  3 is a
> very common situation in Debian, and 4 is important for completeness
> for systems that don't want to use Distribution.Simple, and maybe for
> stuff like libglade?

My feeling is that Cabal's build-depends should deal with dependencies
that it needs to know about in order to build, install and register the
package.  Everything else should be pushed into the OS-level packager.

> I have been thinking that Cabal should worry about 1 and 2 only, and
> let the OS package system worry about 3 and 4,

bingo :)

> but Duncan's
> requirements open the door to 3, and it's not going to be obvious to
> people that they can't talk about tools that Simple doesn't know
> about. In fact, it snuck into the above example.  How the heck does
> Distribution.Simple know what version of Mozilla is installed?  Maybe
> you didn't really mean that?
> 
> But I do think that if we're overhauling the dependency syntax, we
> should think about all kinds of dependencies so we don't paint
> ourselves into a corner.  1 is the only kind that currently has a
> flexible infrastructure; 2 is done now, but can be sloppy sometimes;
> as I haven't always added --with-foo= flags when we need them and
> such.
> 
> But how can we be flexible about 3 and 4?  Are they important?
...
> Am I being too much of a perfectionist in trying to be complete here?
> Should we really just allow 1 and 2 and punt on the rest?  Is it
> important to distinguish between 1 and 2?

They're definitely important when building an OS package from a Cabal
package, and we'd like to automate this process as much as possible.
Therefore it might be useful to have a consistent way to specify hints
to the OS packager, extra dependencies and so on.  This might take the
form of a naming scheme for OS-packaging hint files, or extra sections
in the .cabal file.

I vote for punting on (3) and (4), but allowing hints to be provided for
passing these dependencies to the OS packager.

> Also, I'm not sure we're really doing gentoo a favor by making the
> dependency syntax so complex.  I think Duncan wanted to be able to
> automatically parse the cabal file and figure out the dependencies for
> alex and such, but now he's faced with:
> 
> --enable-mozilla? ( mozilla >= 1.4, [mozilla] ),

This one's easy:  "mozilla? (>=mozilla-1.4)"  (if I got the ebuild
syntax right).  The GHC section reduces to  >=ghc-5.04.  The
--enable-debug section reduces to  "debug? ( HUnit-1.0 )", or goes away
entirely if you don't care about debugging in the OS package.

Basically, the existing build-depends syntax is a subset of what Gentoo
allows, and I'm proposing a superset (modulo the lack of depdencies on C
libraries etc.).  The .cabal->.ebuild generator would have to fail on
dependencies that it didn't understand - I don't think this is a huge
problem, since complex dependencies will be rare, and there will be a
way for the Cabal packager to provide hints to the .cabal->.ebuild
generator.

I don't think we should restrict ourselves to the lowest common
denominator of Gentoo, Debian, FreeBSD, RPM, etc.

Duncan, does this sound plausible?

Cheers,
	Simon