Automated tests for the libraries?

Jeffrey Yasskin jyasskin at gmail.com
Sun Apr 1 02:37:16 EDT 2007


I've been working on getting test running Cabalized and finally have a
patch to send. Extracting tests from the source code is, I think, an
independent problem.

http://jeffrey.yasskin.info/darcs/filepath/Setup.hs has the main
instructions for running the test (with the same test extraction as
the original GenTests.hs), and
http://jeffrey.yasskin.info/darcs/filepath/SingleFileOps.hs has some
operations that I'd like to add to Cabal itself. With some more
experience writing tests like this, we'll definitely want to extract
most of this Setup.hs into a shared module.

The basic idea is that I want to base the tests as much as possible on
production code, but I may need to #define TESTING or some other
symbol in a particular file to get that file's tests to run. So each
file "foo.hs" may be compiled with the 'tag' "foo" (practically, into
the dist/build/foo directory) with -DTESTING, and then its test is run
in the context of ["foo", ""] (look for "foo" object files, and then
the results of `Setup.hs build`). Putting testing builds in a
different place also makes sure that they don't get installed or
released accidentally.

What would be the best way to proceed? Is this system overkill and I
should start over with something simpler (either a single -DTESTING
build tree or just `runhaskell -DTESTING`)? Should I clean up
SingleFileOps and submit it to Cabal? Should I take what I have and
apply it to base?

This patch is based on http://www.cs.york.ac.uk/fp/darcs/filepath/, so
it's a little out of date. Neil, if you're interested in accepting
this for the version of FilePath on darcs.haskell.org, I'll get you a
patch that works there in the next week.

On 3/18/07, Neil Mitchell <ndmitchell at gmail.com> wrote:
> Hi
>
> > > This gives a list of properties, which are automatically extracted
> > > from the documentation, and checked using QuickCheck. All the tool is
> > > in the repo. If anyone wanted to generalise the code in there, it
> > > would be handy!
> >
> > Neil, your extractor looks cool, but I wonder how it'll work with
> > modules like Data.Set whose (commented out) tests define some
> > Arbitrary instances and some helper functions in addition to the
> > actual properties. My first inclination would have been to define the
> > properties in code, perhaps "#ifdef TESTING"ed out, and try to teach
> > Haddock to pull them into the documentation. But yours is definitely
> > simpler and probably worth pursuing until it obviously stops working.
>
> FilePath also has that issue, I do:
>
> module FilePath(
> #ifdef TESTING
>     properties and instances for testing
> #endif
>
>    ) where
>
> Then I just define TESTING when running the test. This has the
> advantage that the tests at least always compile, even as I make other
> changes. In reality, FilePath only requires a couple of things (the
> drive properties) to be exported for testing but not for real use.
>
> I don't see much complication in extending my approach. In fact, I
> suspect that my FilePath test extractor is overly complex, since it
> allows:
>
> Windows: property
> Posix: property
> property
>
> For Windows properties it added System.FilePath.Windows.functions,
> ditto for Posix, and for normal ones it generates the test twice, once
> with Windows and once with Posix. It also detects if a property has
> any free variables, and if not changes it to a single assertion,
> rather than a quickcheck property.
>
> I think it is very handy, and integrating it with Cabal and removing
> the special cases that FilePath requires is probably well worth the
> effort! I would certainly like such a tool...
>
> Thanks
>
> Neil
>


-- 
Namasté,
Jeffrey Yasskin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: cabalize-gentests.dpatch
Type: text/x-darcs-patch
Size: 23246 bytes
Desc: not available
Url : http://www.haskell.org/pipermail/libraries/attachments/20070331/895e2219/cabalize-gentests-0001.bin


More information about the Libraries mailing list