new package description fields, sdist, and preprocessors

[Isaac Jones]

Ross Paterson <ross at soi.city.ac.uk> writes:

> On Wed, Aug 03, 2005 at 07:27:01PM -0500, Brian Smith wrote:
>> On 8/3/05, Ross Paterson <ross at soi.city.ac.uk> wrote:
>> > 1) New field extra-tmp-files, a list of extra files to be removed by
>> >    setup clean, beyond those that can be deduced.
>> 
>> What is the use case for this? Temporary files are either created by
>> tools Cabal already "knows" about (GHC, hsc2hs, etc.), or they are
>> created by hooks. It seems like Cabal should know enough to clean
>> temporary files created by the tools it invokes. If the user writes a
>> hook that generates temporary files, then they should also write a
>> pre/-post-clean hook to delete them.
>
> Indeed, these would be files created by hooks, e.g. the postConf hook
> in defaultUserHooks.  The purpose of the field would be to avoid the need
> for another hook.  For that particular hook (e.g. using an autoconf-based
> configure), it's not feasible to put the temporary files inside dist.

This field sounds fine to me.  I agree with Brian that it may be
preferable to make all the tools generate files into dist/tmp (in
fact, Cabal itself should probably do that more often), but I know
it's not feasible to expect existing tools to change.

>> > 3) New field data-files, a list of files to be copied to a place where
>> >    an executable can find them (e.g. template-hsc.h for hsc2hs):
>> >         Hugs: the directory containing the Main module
>> >         GHC/Windows: the directory containing the executable
>> >         GHC/Unix: /usr/local/share/<exename>
>> >    plus a new function in System.Directory to return the name of this
>> >    directory.  That would address Dimitry's requirements in
>> 
>> How about allowing directories too, which would be copied recursively?
>
> No objection to that.

This sounds good, except that I'm not sure that System.Directory is
the right place for this.  It could also go in Distribution
somewhere, but either way may be fine.

Also, if we do this, we should probably specify the manner in which
such a directory should be layed out so:

1) it doesn't get too cluttered
2) different packages don't stomp on each-other's files and
3) different versions of different packages can use the same filenames.

One stab at it (where dataFileDir :: FilePath) would be that your
datafiles should be in:

System.Directory.dataFileDir
  `joinFilePath` (packageName ++ "-" packageVersion)

and this is where cabal will put it.  Alternately, of course, you'd
have:

dataFileDir :: Distribution.Package.PackageIdentifier -> FilePath
dataFileDir ident = dataFileDirRoot `joinFilePath` (showPackageId ident)

which I like better, but that strongly couples "dataFileDir" to the
Cabal package in that you need to have a PackageIdentifier.  How do
you get that PackageIdentifier?  Well, your program will have to parse
your .cabal file.  No problem! (if you have one)

(snip)
>> BTW, what exactly is a "source distribution?" I think a very
>> reasonable definition is "A source distribution can be built with
>> nothing more than a Haskell compiler/interpreter that supports the
>> required language features and that has the correct packages and
>> libraries installed" In particular, a source distribution would never
>> require any preprocessors.
>
> That wouldn't do -- a source distribution must be system- and
> implementation-independent.  
(snip)

I definitely agree with Ross.  However, I already have a TODO item for
adding a flag to "setup sdist" to handle this in a somewhat more
fine-grained mannar:

** if there's a flag, --include-preprocessed-sources (or something
   better) run the preprocessing phase and include both the
   unpreprocessed and the preprocessed sources in the source tarball?

But really, there are two kinds of preprocessors, as Ross points out.
The kind that produce OS-independent code, and the kind that produce
OS-dependent code.  Perhaps this concept shoudl be added to the
PreProcessor type, and a we could have two flags to sdist:

--include-standalone-preprocessed-sources

Which would generate the OS-independent sources from tools like Alex
and Happy... 

--include-all-preprocessed-sources

Which just includes all of the preprocessed sources as above.

A downside to this is in how it interacts with another proposal to add
tool dependencies.  If a package tool-depends on "alex", and then a
source tarball is created with
--include-standalone-preprocessed-sources, then it actually no longer
tool-depends on alex, so we should regenerate the .cabal file.  I
guess that's no big deal.

What do folks think of this?  Better ieas for the names of the flags?


peace,

  isaac