[Haskell-cafe] Re: GSoC: Improving Cabal's Test Support

[Gregory Crosswhite]

On Apr 6, 2010, at 4:40 PM, Thomas Tuegel wrote:

> Now, if you're telling me I'm going off in the wrong direction by
> proposing to integrate a test framework into Cabal itself, that's
> another story.  Should I pare down my proposal to only include support
> for a proper 'Test' stanza in the package description file and leave
> it at that?

Yes, I think that would be a better use of your time, and furthermore if you arrange this feature so that the programmer can use any test framework that they like, it has the advantage that you are not entrenching a particular way of combining tests from different modules.  If you want there to be communication between Cabal and the test suite, it should be something simple like passing in a location to where the test results should be output (or a flag indicating that they should be written to the screen in "verbose" mode), and having Cabal in return receive from the test runner a single string with a summary of the test results to show to the user (when in "summary" as opposed to "verbose" mode) and possibly a flag if there were any failures so that the user can be specifically alerted to this fact and pointed to the full test report.

Even if all you do is add systematic support for automatically building and running tests automatically as part of the build process, it would still be a major improvement;  it would be especially awesome if you made it possible to separate out the test framework dependencies from the package dependencies, since at the moment people who want to include tests with their package often feel the need to include dependencies on packages such as QuickCheck in order to make life easier for themselves, but if you could modify cabal so it would only pull in such packages when the user requests that tests be run it would be awesome!

Cheers,
Greg