Automated tests for the libraries?

[Jeffrey Yasskin]

On 3/16/07, Neil Mitchell <ndmitchell at gmail.com> wrote:
> Hi Paul
>
> > I think that QuickCheck properties, or at least an abbreviated version
> > of them, should be in the documentation.  See my RangedSet library
> > (http://ranged-sets.sourceforge.net/) for an example of the kind of
> > thing I mean.  However I have to maintain the correspondence between the
> > actual properties and the Haddock comments manually, which is a pain.
>
> See my FilePath library:
> http://www-users.cs.york.ac.uk/~ndm/projects/libraries.php
>
> For example: http://www-users.cs.york.ac.uk/~ndm/projects/filepath/System-FilePath-Version_0_11.html#v%3AreplaceExtension
>
> This gives a list of properties, which are automatically extracted
> from the documentation, and checked using QuickCheck. All the tool is
> in the repo. If anyone wanted to generalise the code in there, it
> would be handy!
>
> Thanks

Neat! I've started http://haskell.org/haskellwiki/Library_tests linked
from Library_submissions to record this stuff. Feel free to rearrange
that of course.

The biggest problem I see with the tests that SimonM is moving is that
they have to be run inside a built GHC source tree, which makes it
hard to work just on the libraries or test them on another compiler.

Neil, your extractor looks cool, but I wonder how it'll work with
modules like Data.Set whose (commented out) tests define some
Arbitrary instances and some helper functions in addition to the
actual properties. My first inclination would have been to define the
properties in code, perhaps "#ifdef TESTING"ed out, and try to teach
Haddock to pull them into the documentation. But yours is definitely
simpler and probably worth pursuing until it obviously stops working.

It would be nice to use Cabal to run these tests since it already
knows how to pick a compiler. I'll start on a runTests hook using
GenTests.hs.

-- 
Namasté,
Jeffrey Yasskin