working on a fix for ticket #212

[Laszlo Nagy]

Hi All,

Trying to create unit test suite.. work in progress. Comments are more
than welcome!

http://sites.google.com/site/rizsotto/patches/cabal-ticket-212-v1.darcs-send


As an easy start, I wrote unit tests with QuickCheck on the following files:

 Distribution/License.hs
 Distribution/Package.hs
 Distribution/Compiler.hs
 Distribution/System.hs
 Distribution/ModuleName.hs
 Distribution/PackageDescription.hs

I wrote simple properties like this, for all data type

> prop_parse x = x == parse $ display x

I did merged the existing QuickCheck test cases into one file. I found
it easier to reuse the generator functions for different types.

My findings:
 - The System.Platform did not passed parse tests. The reason: both
System.OS and System.Arch allowed the dash character in the name. But
System.Platform text representation was concatenate the two strings
separated by a dash. So I just simple removed the dash from them.
Solved! ;)
- VersionRange does not passes parse tests, iff more than two ranges
is present. As far as I could see, the parse method is wired to handle
only two ranges. Not corrected, because found no good solution. Need
help on this!
- Package.Dependency also not passes parse tests. (It is depend on
VersionRange success.)
- Compat.ReadR gather property is obsolete and does not compile. Can't
figure it out how could I rewrite it. Need help!
- PackageDescription does not passes simple parse test. I guess the
problem is with the test itself! ;) I used showPackageDescription
method to generate string representation of this type and parse it
back. I could not judge it is an expected behaviour or a bug. Need
some explanation.


My problem is mainly how to run these tests? As unit tests they are
pure, does not manipulate any files on disks, does not open socket or
anything. (Michael Feathers use the unit test word as strict as this.)
So, I expected to run them by anyone, who could compile the sources.

#1 'cabal test' could be the best way to exercise them. But the user
hook is so complicated for a simple 'compile and run'. Found others
ignoring the arguments and call System.Cmd, execute _the compiler_.
The compiler name is wired and dependencies also not visible enough,
not to mention the extension handling. Basically compile by hand is
against cabal! :) (Pro: the best way to invoke; Con: very hard to do)

#2 Create testing flag in the cabal file, which enable an executable
to be build. (Xmonad guys doing this way.) It compiles easy, developer
need not worry about different compilers. Running the test can be done
by hand (or darcs) or the test user hook can execute it. If the main
product is a library, then it does not link against that, since the
library is not installed. Instead developers specify the source dir
and it recompiles the tested sources. (Pro: easy to do; Cons: does not
test against library)

#3 Create another cabal file. Roughly speaking it is same as #2. In my
opinion it is suite best for functional tests. (Pro: easy to do; Cons:
testing target must be installed first)

#4 Use the QuickCheck script. Based naming convention this script runs
through the sources, 'grep' the properties, and exec an interpreter to
run them. (Pro: easy; Cons: interpreter name is wired in the script)

#5 As a proposed long term solution: I might add a new parameter to
the 'executable' region describe the target is a unit test. Compile
(and run) it only on 'cabal test' invocation. (Con: It does not exists
and might be hard to implement :))

Thanks for any help!

Regards,
Laszlo