cabal, UHC, and different backends

Thu Oct 21 03:57:40 EDT 2010

Hi All,

so, the design for the interface to a compiler would be more or less like this:
- type Way = String. Cabal is agnostic of any meaning of a Way.
- Supported compilers provide the meaning of a Way, similar to compiler specific 'build executable' etc functions:
  <Compiler>.wayToFilePath :: Way -> FilePath
  <Compiler>.wayToFlags :: Way -> [String]
- Compilers also specify when to use the above. Although it may be desirable to let Cabal do all the stuff depending on a Way, I think
  - it is a lot of work, actually it may be so much work that it will become a stumbling block,
  - it will be tricky, because it also would include Way specific runtime system, libraries, etc,
  - it will take time for compilers to adapt.
  So I'd prefer to identify a set of points where Cabal does/doesn't do the Way specific stuff. This allows per compiler choice and a more gradual path of adaption. For example:
  - location of .hi files, library files, etc (this is the main part), which would boil down to passing a destination location which does or does not include 'wayToFilePath'.
  - location/construction of package descriptor file.

For dealing with constraints and the cabal language specification, a Way is (should be?) orthogonal to other constraints. That is, for a Way neutral package, the same dependencies would hold for all Ways. Conceptually, a constraint then is a set/map of constraints, one for each Way, only to be strictly smaller than the largest possible set when a particular Way is not supported. The cabal language would then require 'member' and 'filter' like mechanisms. I am sure I simplify matters (too much? :-)).

As for using Nix ideas, this will require a different build system inside Cabal. The basic idea is simple, every artifact is a function result, no side effects. Making it work when no tool behaves without side effects (files are always put into a file system), etc etc, is more difficult, most likely impossible without compromises. I have thought of ways of doing this inside and/or with UHC, but maybe something along the lines of recent work of Neil Mitchell would be an easier starting point.

cheers,

On  20 Oct, 2010, at 01:05 , Duncan Coutts wrote:

> Hi Atze,
> 
> btw, I'm cc-ing the cabal-devel list since this is relevant for the
> Cabal specification, and other people may like to chime in on the issue.
> 
> On Tue, 2010-10-19 at 10:13 +0200, Atze Dijkstra wrote:
>> On  18 Oct, 2010, at 23:38 , Duncan Coutts wrote:
>> 
>>> 
>>> So the approach I would like to take is to have a proper notion of
>>> "way". This term comes from GHC where profiling, normal, etc are all
>>> different "ways". NHC had several different profiling ways. I think the
>>> notion can be generalised and supported in Cabal. It should cover both
>>> things like the same backend but ABI-incompatible options (like
>>> profiling) and also entirely different backends. Cabal would want to
>>> track each package separately for each way. It would need to know which
>>> ways are compatible/incompatible so it knows if it can mix packages of
>>> different ways (backend obviously cannot, but some forms of profiling or
>>> parallel vectorisation are ABI compatible and are expected to be mixed).
>>> 
>>> So yes we have a notion of "the right thing to do" but there's no
>>> current code in Cabal for doing this. Simon has also recently been
>>> interested in the issue of tracking profiled libs separately from
>>> normal. So perhaps we can cooperate on a detailed design.
>> 
>> I intend to be at BelHac, we can then spend some time on it. However
>> for kicking off immediately:
>> - There are (at least) 2 dimensions in which a compiler must be able
>> to vary: target + variation, so I guess that makes type Way =
>> (Target,Variation). Possibly a third: Platform, when cross compilation
>> would come into play (like jhc does).
> 
> The way I'd like to handle this at the level of the Cabal spec (which is
> supposed to be mostly compiler-agnostic) is to say that the way field in
> the installed package info is just a string. This is the same as the new
> installedPackageID field which is also a string (and distinct from the
> source package id). This allows different compilers to encode different
> information. For example for hugs and nhc the installedPackageID is a
> string containing the name and source version, but for ghc it also
> contains an ABI hash.
> 
> For the way, we can do something similar. UHC can encode its target,
> variation etc and other compilers can do different things. The main
> point is to distinguish different ways. Then interpreting which ways are
> compatible is compiler-dependent.
> 
>> - UHC currently organises its internal directory structure like
>> <pkg>/<compiler-variant>/<target>/<variation>, of which
>> <compiler-variant> is UHC specific, but something like this is needed
>> to distinguish. This is something we could standardize on, that is,
>> cabal assumes a fixed way of mapping Way to FilePath.
> 
> Right, I suspect the standard layout will still be compiler-dependent
> but for several compilers it may include the way.
> 
>> - An issue is whether a compiler knows of such a convention, or not?
>> UHC currently knows for which target + variation it is compiling and
>> adapts its location for generating package content. This could be
>> factored out, or not. I am not sure what is best, although factoring
>> out would make a compiler rely more on cabal, make compiler and cabal
>> more intertwined. However, my guess is, that this part has the
>> greatest impact on cabal.
> 
> Ideally, I would like the compilers to have no hard coded knowledge of
> the layout convention. Instead the compilers should just use the
> installed package info. This is what GHC does for example. In future I
> would like to modify the default layout that cabal-install uses for
> installed packages to use a Nix-style persistent package store.
> 
> Also, distributions and other custom deployments typically want more
> control over the layout. Of course people doing this have to be aware of
> the constraints.
> 
>> - As 1 end of this design space, when a compiler knows about these
>> locations, changes to cabal can mostly be avoided. For UHC I ran cabal
>> with a --uhc-option=-tjscript for specifying a different target, and
>> all went well apart from the generation of a package descriptor file.
> 
> So, since the job of a package manager does involve knowing a fair bit
> about the layout and organisation of installed packages, I would like to
> see cabal know something about the "way", just so that it can allow
> multiple instances of the same package with different ways. The package
> registration info would also contain the way.
> 
> In any case, cabal needs to know the way to work out which dependencies
> can correctly be combined together.
> 
>> If this would also be done by UHC, cabal's only job (??) would be to
>> pass the appropriate flags around and allow a .cabal file have
>> constraints on target etc.
>> - .cabal files need a way to refer to targets etc for constraints
>> (i.e., package only compiles for certain backend).
> 
> Hmm. I'll think about that. Can it perhaps be expressed as a language
> restriction?
> 
>> - Should targets and variations be standardized upon, i.e. data Target
>> = ...? Or just allow any String to act as a Target, i.e. type Target =
>> String?
> 
> As I mentioned above, the information in the way will be different for
> different compilers, so I think string is appropriate.
> 
> Duncan
> 
> _______________________________________________
> cabal-devel mailing list
> cabal-devel at haskell.org
> http://www.haskell.org/mailman/listinfo/cabal-devel

                - Atze -

Atze Dijkstra, Department of Information and Computing Sciences. /|\
Utrecht University, PO Box 80089, 3508 TB Utrecht, Netherlands. / | \
Tel.: +31-30-2534118/1454 | WWW  : http://www.cs.uu.nl/~atze . /--|  \
Fax : +31-30-2513971 .... | Email: atze at cs.uu.nl ............ /   |___\