Fwd: [Haskell-cafe] Data.Tree.Zipper in the standard libraries

[Duncan Coutts]

On Sat, 2008-05-31 at 10:10 -0700, Don Stewart wrote:

> You're on your own on this one, Ian. QuickCheck is a cheap, simple way
> to get good coverage, that's why Haskell library writers use it.
> 
> Please include tests, Krasimir!

Ian does have a point. We're talking about two different testing
contexts:

      * GHC nightly builds
      * tests run on commits by developers

I absolutely agree with you Don about the second context. That's the one
eg for bytestring where we found lots of our own stupid mistakes before
they caused segfaults for end users. It vital for initial development
and for making subsequent changes to the library.

On the other hand Ian is quite right that for GHC nightly builds (where
you're not actually making any changes to the module) it does seem like
a lot of wasted effort to run the same tests again and again.

Perhaps we can see a way to run the right tests in the right context. If
there have been changes then the full QC testsuite is demanded. If on
the other hand we're just looking at a system level integration test
then perhaps we do not need to run all the tests at a maximum level of
coverage.

Perhaps we could take a probabilistic approach. How about a QC test
driver that runs say 2 instances of each QC property at the same QC
Aribtrary 'size' as you'd hit when you do the full 100 tests. That way
each nightly run tests a different sample and we don't just keep hitting
the 0/[] cases (though we will still hit 0/[] sometimes, the 'size'
parameter is a maximum not minimum). It should still catch the obvious
integration bugs (like changed semantics of imported modules). Then we
can do 'full' runs at important junctures like release candidates.

Duncan