[Haskell-cafe] Re: GSoC: Improving Cabal's Test Support

[Duncan Coutts]

On Wed, 2010-04-28 at 10:42 -0400, Thomas Tuegel wrote:
> On Wed, Apr 28, 2010 at 4:54 AM, Duncan Coutts
> <duncan.coutts at googlemail.com> wrote:
> > On 28 April 2010 09:24, Richard G. <richardg at richardg.name> wrote:
> >> I think that formatted plain-text output would be much better than XML,
> >> something that is human-readable and relatively easy to parse via machine.
> >>  Something similar to the GHC error output would work well because
> >> developers are familiar with it.
> >
> > I have previously advocated a library interface as a detailed
> > testsuite interface (in addition to a lowest common denominator
> > interface of stdio+exitcode). That is a test stanza in a package
> > .cabal file would specify a module containing an entry point of the
> > right type (like main but using a more interesting type).
> >
> > That way, cabal or any other tool could run the testsuite and produce
> > results in whatever format it likes.
> 
> I appreciate the elegance of this method, but it seems to me that it
> requires dynamic loading, which is currently in a sorry state.

Actually it doesn't require dynamic loading. It just requires compiling
a stub program that imports the user's library and some test-runner
code. Cabal is good at doing that kind of thing already (eg Setup.hs
scripts).

> One way or another, cabal will need to provide a data structure it expects
> test suites to use for results.  Is there a substantial advantage to a
> library interface, versus providing Read/Show instances for the test
> result data structure?

Yes, it means the testing agent (cabal-install or some other
program/system) can do more than simply run all the tests. It means it
can enumerate them and not run them (think a GUI or web interface), run
a subset of tests, run them in parallel etc.

Duncan