Custom preprocessor with custom file includes

[Jeroen Bransen]

Hi all,

So, any feedback for this? I still think it is a good idea to move the filetime change check for a preprocessed file from preprocessFile to mkSimplePreProcessor (both in Distribution.Simple.PreProcess). This gives preprocessors the flexibility to define their own dependency tracking, and would solve the problem with multiple input files generating a single Haskell file. The change is quite simple, I could do it, but I'd like some feedback on this.

In addition to this, we could add something like:

mkAdvancedPreProcessor :: (FilePath -> FilePath -> Verbosity -> IO (), FilePath -> Verbosity -> IO [FilePath])
                      -> (FilePath, FilePath)
                      -> (FilePath, FilePath) -> Verbosity -> IO ()

which, in addition to the simplePreProcessor, should also provide a function that returns all dependencies of a certain file. Then the file time change check can also be done for the cases where multiple files generate a single Haskell file. But that's maybe a less generic case which could be handled in the preprocessor itself.

Regards,
Jeroen Bransen

Op 5 nov 2012, om 17:14 heeft Jeroen Bransen het volgende geschreven:

> With Distribution.Simple.PreProcess.PreProcessor we can add a custom preprocessor to our Haskell project. This works fine when there is a one-to-one mapping from other-source to Haskell-source file, but there is no good support for other types of preprocessors. For example, for our UUAGC [1] project we also have a cabal plugin [2], and we have the following situation.
> 
> In general, an AG file (.ag) is preprocessed into a single .hs file with the same name. However, AG files can include other AG files so typically multiple AG files are preprocessed in a single .hs file. Let's say we have a X.ag which includes Y.ag and Z.ag, and from this X.hs should be generated. The first time that we build this is fine, as cabal sees that it can generate X.hs from X.ag and it calls the custom preprocessor. However, if we now edit Y.ag and rebuild, nothing will happen! Cabal will only check the file time of X.ag, which is not changed, so decides that X.hs does not need to be re-generated.
> 
> Currently we have solved this in a quite ugly way by means of a buildHook that removes the generated .hs file whenever this file should be rebuild. Although this works it's of course not a good solution, but with the current setup I see no other way of doing this. I believe this should be a more common situation and it should actually be up to the preprocessor to decide whether or not to rebuild a file. I suggest the following change to Distribution.Simple.PreProcess.
> 
> In the current version, the check for rebuilding is in preprocessFile, which simply compares file change times of input/output files. I suggest to remove that check from there and always call the preprocessor. It is then up to the preprocessor to decide not to preprocess a file when the generated file is already in place and nothing has changed. To handle the common situation where there is a one-to-one mapping, I suggest to move this check to mkSimplePreprocessor which is probably used by almost all preprocessors. In that way nothing changes for 'simple' preprocessors, but there is some extra flexibility for the cases like UUAGC.
> 
> Regards,
> Jeroen Bransen
> 
> 
> [1] http://hackage.haskell.org/package/uuagc
> [2] http://hackage.haskell.org/package/uuagc-cabal