more automatic tests for containers

[Joachim Breitner]

Hi,

Am Mittwoch, den 21.12.2011, 13:31 +0900 schrieb Kazu Yamamoto:
> Especially to Johan and Milan,
> 
> I'm thinking to introduce more automatic tests for the containers
> library. Let's take Data.Map as example:
> 
> 1) doctest
> 
>   Some haddoc documents include the following style.
> 
>   -- > Data.Map.null (empty) == True
> 
>   If we modify it like
> 
>   -- >>> Data.Map.null (empty)
>   -- True
> 
>   we can use doctest. And we can remove corresponding unit test
>   from test/*-properties.hs.

I’m not sure what the benefit is here. containers is a “high profile”
library which sees lots of effort. Introducing a new testing library
(doctest) just to avoid having to add those test cases to the test suite
does not seem to gain much, but takes away some flexibility while
writing good documentation. Maybe the “==” form is more readable to some
users?

> 2) test-framework-th
> 
>    If we replace "test_" with "case_", we can use test-framework-th to
>    generate the long list of test cases. We don't need to update it when
>    we add "prop_" and "case_".
> 
>    We can specify the number of QuickCheck test as follows:
> 
>    runghc -i.. -DTESTING map-properties.hs --maximum-generated-tests=500
> 
> Since this is test stuff, we don't have to add doctest and
> test-framework-th to containers.cabal.
> 
> What do you think? If you like this, I will work and send you pull
> request.

For the same reason as above, I don’t think it is worth it. Maintaining
the lists of cases is not nice, but the amount of work is not high in
relation to the efforts going into the library, so the gain is litte.
The downside is that the test suite cannot be run on architectures where
Template Haskell is not supported. Given that containers does some
bitshifting that can exhibit architecture dependent bugs, this could put
us in the bad situation that the tests cannot easily be run on those
machines that need the tests the most.

If you would like to work on the test suites, maybe you can identify
parts of the code that are under-tested and come up with test cases?

Greetings,
Joachim


-- 
Joachim "nomeata" Breitner
  mail at joachim-breitner.de  |  nomeata at debian.org  |  GPG: 0x4743206C
  xmpp: nomeata at joachim-breitner.de | http://www.joachim-breitner.de/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part
URL: <http://www.haskell.org/pipermail/libraries/attachments/20111221/f8fbd819/attachment-0001.pgp>