haddock-2.3.0 literate comments discarded from .lhs input

[Duncan Coutts]

On Thu, 2009-05-28 at 23:40 +0100, Claus Reinke wrote:
> >> If you don't want to move from absolute paths for non-core packages,
> >> the current system should just work, right?
> > 
> > Yes.
> 
> The current system being the $topdir one.

Yep. It works, it's just not nice, it's ghc-specific and only make sense
when ghc is installed in a prefix-independent way.

> > Though it also allows for the possibility of relocatable sets of
> > packages that are not installed relative to the compiler. But more
> > importantly it's more general and simpler than the current '$topdir'
> > that ghc uses.
> 
> 'it' now being the new system evolving in this thread, or have I missed
> anything?

The new system I've been proposing.
 
> >> (a) making ghc-pkg (optionally) instantiate any variables in its
> >> database in (all of) its command-line output and 
> > 
> > Yes, though I'm only asking for two vars (previously one), not an ad-hoc
> > set of vars.
> > 
> >> (b) allowing non-core packages to be relocated without having to
> >> update ghc-pkg's database.
> > 
> > In my suggested system this is possible if that set of packages use
> > their own package db (containing relative paths).
> 
> That is news to me - was that specified before this thread moved
> to ghc-users?

It was in the first email that was cc'ed to ghc-users:

        How about this: a way to specify paths in the package
        registration info that are relative to the location of the
        package db they are in. That makes sense beyond just ghc and
        even with would allow other sets of relocatable packages, not
        just those installed with ghc.

> > In your system it's possible by updating some var in a central registry
> > and having that set of packages use paths relative to that var.
> 
> So, essentially, your system would have to keep a file listing the
> various package.conf locations (currently, GHC only knows about
> two: system/user, everything else would have to be passed on the
> commandline..). While my system would have to keep a file listing
> the variable bindings, so that tools processing the package db can
> instantiate the variables.

If you want multiple relocatable sets of packages that are immediately
"available" in the environment.

> I could see both approaches being useful, even together.
>  
> > So ghc's current system uses two vars, $topdir and $httptopdir. 
> 
> This is GHC's view of its database. It should be useable independently,
> via ghc-pkg and ghc api clients (such as GHC, GHCi, Haddock, ..) -
> all of which should be able to resolve the variable bindings, in the
> same way.

It's not usable independently, ghc does not always have a topdir. This
makes life hard for tools. It's also not clear what topdir would mean in
the context of other compilers.

> Btw, it would really be nice if the package handling code 
> was shared rather than duplicated.

It would be nice, yes.

> > I'm proposing to replace those with a standardised ${pkgroot} and
> > ${pkgrooturl} vars which are usable by all compilers and in more
> > situations.
> 
> Now you are talking about Cabal's view of its database. 

Cabal does not own the package databases, however it does expect that
they are in the format describe by the Cabal spec, which places
obligations on Haskell implementations to be somewhat package-aware.

> It doesn't have to expose the underlying implementation's view,
> especially since the other implementations organise their package
> handling differently.

All compilers use the same information (it's in the Cabal spec). They do
store it differently but they all identify the location of the
information using a file path. That seems pretty universal, compared to
$topdir.

> And why just two variables? Is $pkgroot about .hi files, .a/.so./.dll
> files, or about include files, or haddock indices, or ..?

You only need one variable to identify the location of the installed
package description. All relative paths can be constructed from that.
The second variable is to allow for two representations of the same
location, one as a native system path, the other as a URL. We do not
need different variables for different kinds of files (except in as much
as some fields use paths and some urls).

> In windows, these tend to end in a common sub-hierarchy, but you're
> aiming for something general, right?

If you're making a relocatable package then these files will be in a
common sub-hierarchy and you would use relative paths. If you're not
making a relocatable package (eg following the Linux FSH) then you would
not use relative paths.

So that should be general. It does not remove any existing capability
and it adds the ability to have relative paths for relocatable packages.

Perhaps what you're saying is that we should be able to take any package
whether it lives in a common sub-hierarchy or not and relocate it. In
general this is problematic since packages can embed paths and if those
paths are not relative to a common root then you have to specify them
all (Cabal enables this by setting environment variables). Assuming
that's ok, then even this rare use case is still possible, just by
editing the package registration information. It doesn't need to be
"simplified" by having one var per package entry.

> > You're proposing a central registry of vars and to have ghc-pkg
> > (optionally) expand these vars which could be used anywhere in the
> > installed package descriptions. Presumably you're also suggesting some
> > mechanism to query and update this registry of variables.
> > 
> > Is that a fair summary?
> 
> I think so. And you're proposing several separate registries (hasn't
> that been a Cabal problem in the past, even with just user and system
> to choose from?).

Cabal can be instructed to use a specific package db, not just global
and user.

> Presumably you're also suggesting some mechanism
> to query and update the meta-registry of package database locations.

I wasn't actually proposing a meta-registry (ghc already supports this
via an environment variable) but it is a possible extension which brings
it closer to the capabilities of your proposal.

My proposal is just to replace the use of $topdir with something that
every compiler can implement and which tools can understand. The fact
that it could be extended without having to modify the installed package
description format or the tools which understand that format is a bonus.
I would not argue particularly for or against such an extension, my main
concern is for a simple clear spec and the sanity of tool authors.

Duncan