cabal design

[Duncan Coutts]

On Mon, 2005-08-15 at 08:52 -0700, Isaac Jones wrote:
> Frederik Eaton <frederik at a5.repetae.net> writes:
> 
> > 4) Cabal's inability to let users specify package.conf-s other than
> > the user or global package.conf
> 
> I think the gentoo guys were going to implement this for us; in any
> case, it's a subset of 1).  This should be pretty easy; the only
> question is whether we should have a --in-place or a
> --package-conf=... Where in-place would choose a "standard" one
> (./packages.conf or something).  The reason I'd prefer the former is
> that not all systems _have_ an idea of a package.conf file, so
> --package-conf= would make no sense for most compilers.  I try to
> avoid that.  The reason for the later would be if there's some
> use-case that --in-place does not work for.

So ./setup register --in-place suggests that the library will be
registered in it's current location, ie the build tree.

So this would allow the library to be used in building other libraries
or programs in the same "distributable".

Or it would allow the library to be used without installing to its final
location which might be useful in some circumstances (which I must admit
are not obvious to me).

So this is a useful feature and we're happy to help out in implementing
it.

However what we the gentoo folk want is subutly different. We want to
register into a package database other than the global or user one.
However we want to register it for it's final installed location, not
the current location in the build tree.

Actually, it's not that we really want this but it would be sufficient
for our needs. For the sake of innocent bystanders let me recap on a
discussion the gentoo packaging people have been having with Isaac over
the past few weeks:

The way packages are built for a distribution (like gentoo, debian,
fedora etc) is slightly different from the way you normally build and
install a package. We configure and build and install the package in a
"sandbox", that is in a temporary directory and with permissions set up
such that there is no write access to the "live" filesystem. In
particular the instllation phase is also done to a temporary directory.
This gives us an "image" of the package. Now in a binary packaging
system that image gets compressed into a binary package file (eg a .deb
or a .rpm). With gentoo we skip that step and just merge the image of
the package into the filesystem. And note that this marging phase is
totally under control of the package manager and the package's build
system get's no control. This is partly a security measure; the
package's build system only gets to run in the sandbox and so never has
write access to the live filesystem.

So this is why people who do packaging like packages that can be
installed just by copying files and without having to modify anything
else on the target system - like registering. For example, many systems
which need to reigister clients (eg cron, pam, apache modules etc) have
been converted to use directories of config files rather that having to
modify a single global file since the former allows one file to be owned
per package.

Packages can have post install actions. So for cabal packages, this is
the obvious place to put the registering phase. However we can't just
run ./setup register. There are a couple reasons for this, one is that
the build directory doesn't actually exist anymore since it gets cleaned
before merging (this is not unreasonable since we can also build binary
packages and move them to another machine in which case the build
directory is not available either). Also it would be against the
packaging policy to run an untrusted program (the cabal setup program)
outside of the sandbox.

So this is why cabal provides the ./setup register --gen-script feature.
It produces a shell script that performs the registration. It means all
you need is the script rather than the setup program and any other bits
of build tree that the setup program uses. However this is still a
violation of the packaging policy since we're running a script produced
by an untrusted program outside of the sandbox. From a security point of
view it's preferable to have the data and run trusted actions on that
data rather than be given the actions that know what to do with the
data. The downside from the cabal point of view is that just giving out
the data is against the spirit of the abstraction that cabal is trying
to provide.

However the package manager needs to do more things than the cabal
abstraction currently provides hence the current pressure to "just give
me the data!". The --gen-script feature allows us to register but we
also need to be able to unregister and re-register. And we need to be
able to do more complicated things like unregistering an old version
*after* registering a new version without accidentally unregistering
that new version.

Now these are not features that we expect cabal to provide (since they
are peculuiar to package management systems and probably vary from one
to another - especially the idea of installing a new version of a
package before uninstalling an old version of a package). However if we
are to be able to implement these features (and we must otherwise ghc's
package database will be out of sync with the installed packages) then
we need some way of getting at the collection of packages that each
cabal distributable provides. Then we can take responsability for
registering, unregstering and re-registering.

[In case anyone's even remotely interested in the problem of "uninstall
after install", here it is: suppose the gentoo package "foo-1.0"
provides a cabal package "Foo-1.0". Now we fix a bug or something and
release a new revision "foo-1.0-r1" which still provides the cabal
package "Foo-1.0". Then if we were to unregister the packages provided
by foo-1.0 after registering the ones provided by foo-1.0-r1 then we
would end up unregistering Foo-1.0 completely. Instead we need to take
the set difference of the packages provided by the new and old versions
and only unregister those. If we could register/unregister packages just
by dropping files in a package database directory then this would all
"Just Work"tm but that's sadly not the case. So we need to do more
complicated things.]

So back to the original topic... we wanted to implement a feature that
would enable us to get at the packages that need to be registered. So we
were hoping to subvert the ./setup register --in-place idea. However as
I said at the beginning it doesn't quite do what we want. We're happy to
help with implementing this useful feature but we're also still looking
for a nice comprimise that will allow use to implement the features we
need for proper package management while at the same time respecting the
need to maintain a sane interface to cabal.

Duncan