Automated tests for the libraries?

[Neil Mitchell]

Hi

> > This gives a list of properties, which are automatically extracted
> > from the documentation, and checked using QuickCheck. All the tool is
> > in the repo. If anyone wanted to generalise the code in there, it
> > would be handy!
>
> Neil, your extractor looks cool, but I wonder how it'll work with
> modules like Data.Set whose (commented out) tests define some
> Arbitrary instances and some helper functions in addition to the
> actual properties. My first inclination would have been to define the
> properties in code, perhaps "#ifdef TESTING"ed out, and try to teach
> Haddock to pull them into the documentation. But yours is definitely
> simpler and probably worth pursuing until it obviously stops working.

FilePath also has that issue, I do:

module FilePath(
#ifdef TESTING
    properties and instances for testing
#endif

   ) where

Then I just define TESTING when running the test. This has the
advantage that the tests at least always compile, even as I make other
changes. In reality, FilePath only requires a couple of things (the
drive properties) to be exported for testing but not for real use.

I don't see much complication in extending my approach. In fact, I
suspect that my FilePath test extractor is overly complex, since it
allows:

Windows: property
Posix: property
property

For Windows properties it added System.FilePath.Windows.functions,
ditto for Posix, and for normal ones it generates the test twice, once
with Windows and once with Posix. It also detects if a property has
any free variables, and if not changes it to a single assertion,
rather than a quickcheck property.

I think it is very handy, and integrating it with Cabal and removing
the special cases that FilePath requires is probably well worth the
effort! I would certainly like such a tool...

Thanks

Neil