cabal, UHC, and different backends

[Duncan Coutts]

Hi Atze,

btw, I'm cc-ing the cabal-devel list since this is relevant for the
Cabal specification, and other people may like to chime in on the issue.

On Tue, 2010-10-19 at 10:13 +0200, Atze Dijkstra wrote:
> On  18 Oct, 2010, at 23:38 , Duncan Coutts wrote:
> 
> > 
> > So the approach I would like to take is to have a proper notion of
> > "way". This term comes from GHC where profiling, normal, etc are all
> > different "ways". NHC had several different profiling ways. I think the
> > notion can be generalised and supported in Cabal. It should cover both
> > things like the same backend but ABI-incompatible options (like
> > profiling) and also entirely different backends. Cabal would want to
> > track each package separately for each way. It would need to know which
> > ways are compatible/incompatible so it knows if it can mix packages of
> > different ways (backend obviously cannot, but some forms of profiling or
> > parallel vectorisation are ABI compatible and are expected to be mixed).
> > 
> > So yes we have a notion of "the right thing to do" but there's no
> > current code in Cabal for doing this. Simon has also recently been
> > interested in the issue of tracking profiled libs separately from
> > normal. So perhaps we can cooperate on a detailed design.
> 
> I intend to be at BelHac, we can then spend some time on it. However
> for kicking off immediately:
> - There are (at least) 2 dimensions in which a compiler must be able
> to vary: target + variation, so I guess that makes type Way =
> (Target,Variation). Possibly a third: Platform, when cross compilation
> would come into play (like jhc does).

The way I'd like to handle this at the level of the Cabal spec (which is
supposed to be mostly compiler-agnostic) is to say that the way field in
the installed package info is just a string. This is the same as the new
installedPackageID field which is also a string (and distinct from the
source package id). This allows different compilers to encode different
information. For example for hugs and nhc the installedPackageID is a
string containing the name and source version, but for ghc it also
contains an ABI hash.

For the way, we can do something similar. UHC can encode its target,
variation etc and other compilers can do different things. The main
point is to distinguish different ways. Then interpreting which ways are
compatible is compiler-dependent.

> - UHC currently organises its internal directory structure like
> <pkg>/<compiler-variant>/<target>/<variation>, of which
> <compiler-variant> is UHC specific, but something like this is needed
> to distinguish. This is something we could standardize on, that is,
> cabal assumes a fixed way of mapping Way to FilePath.

Right, I suspect the standard layout will still be compiler-dependent
but for several compilers it may include the way.

> - An issue is whether a compiler knows of such a convention, or not?
> UHC currently knows for which target + variation it is compiling and
> adapts its location for generating package content. This could be
> factored out, or not. I am not sure what is best, although factoring
> out would make a compiler rely more on cabal, make compiler and cabal
> more intertwined. However, my guess is, that this part has the
> greatest impact on cabal.

Ideally, I would like the compilers to have no hard coded knowledge of
the layout convention. Instead the compilers should just use the
installed package info. This is what GHC does for example. In future I
would like to modify the default layout that cabal-install uses for
installed packages to use a Nix-style persistent package store.

Also, distributions and other custom deployments typically want more
control over the layout. Of course people doing this have to be aware of
the constraints.

> - As 1 end of this design space, when a compiler knows about these
> locations, changes to cabal can mostly be avoided. For UHC I ran cabal
> with a --uhc-option=-tjscript for specifying a different target, and
> all went well apart from the generation of a package descriptor file.

So, since the job of a package manager does involve knowing a fair bit
about the layout and organisation of installed packages, I would like to
see cabal know something about the "way", just so that it can allow
multiple instances of the same package with different ways. The package
registration info would also contain the way.

In any case, cabal needs to know the way to work out which dependencies
can correctly be combined together.

>  If this would also be done by UHC, cabal's only job (??) would be to
> pass the appropriate flags around and allow a .cabal file have
> constraints on target etc.
> - .cabal files need a way to refer to targets etc for constraints
> (i.e., package only compiles for certain backend).

Hmm. I'll think about that. Can it perhaps be expressed as a language
restriction?

> - Should targets and variations be standardized upon, i.e. data Target
> = ...? Or just allow any String to act as a Target, i.e. type Target =
> String?

As I mentioned above, the information in the way will be different for
different compilers, so I think string is appropriate.

Duncan