working on a fix for ticket #212

[Laszlo Nagy]

Hi Steve,

I saw Your patch. The existing unit tests (on parsing) does not cover
enough all the cases, and instead write more by hand thought
QuickCheck could suite better for this job. And my findings prove me.
;) The test cases could be run by the test-framework without any
problem I guess. And I already tried it.. Based on the experience I
wrote the enumeration in my previous letter. (The test harness require
to install the Cabal first, and just after that it compiles against
Cabal. I think installation should not be a precondition, but accept
it, since I could not offer a better working solution.)

Regards,
Laszlo


On Wed, May 13, 2009 at 11:59 PM, Stephen Blackheath [to cabal-devel]
<rubbernecking.trumpet.stephen at blacksapphire.com> wrote:
> Laszlo,
>
> I have done the same thing!  I posted a patch that includes a simple
> test harness using the test-framework package, which allows QuickCheck
> and HUnit tests to be mixed.  Would you consider using that instead,
> because my tests are HUnit-based?
>
> It is not integrated with 'cabal test', though.  Here's my patch:
>
> http://upcycle.it/~blackh/cabal/cabal-ticket-89-v5.darcs-send
>
> Duncan Coutts is evaluating this patch for me at the moment.
>
>
> Steve
>
> Laszlo Nagy wrote:
>> Hi All,
>>
>> Trying to create unit test suite.. work in progress. Comments are more
>> than welcome!
>>
>> http://sites.google.com/site/rizsotto/patches/cabal-ticket-212-v1.darcs-send
>>
>>
>> As an easy start, I wrote unit tests with QuickCheck on the following files:
>>
>>  Distribution/License.hs
>>  Distribution/Package.hs
>>  Distribution/Compiler.hs
>>  Distribution/System.hs
>>  Distribution/ModuleName.hs
>>  Distribution/PackageDescription.hs
>>
>> I wrote simple properties like this, for all data type
>>
>>> prop_parse x = x == parse $ display x
>>
>> I did merged the existing QuickCheck test cases into one file. I found
>> it easier to reuse the generator functions for different types.
>>
>> My findings:
>>  - The System.Platform did not passed parse tests. The reason: both
>> System.OS and System.Arch allowed the dash character in the name. But
>> System.Platform text representation was concatenate the two strings
>> separated by a dash. So I just simple removed the dash from them.
>> Solved! ;)
>> - VersionRange does not passes parse tests, iff more than two ranges
>> is present. As far as I could see, the parse method is wired to handle
>> only two ranges. Not corrected, because found no good solution. Need
>> help on this!
>> - Package.Dependency also not passes parse tests. (It is depend on
>> VersionRange success.)
>> - Compat.ReadR gather property is obsolete and does not compile. Can't
>> figure it out how could I rewrite it. Need help!
>> - PackageDescription does not passes simple parse test. I guess the
>> problem is with the test itself! ;) I used showPackageDescription
>> method to generate string representation of this type and parse it
>> back. I could not judge it is an expected behaviour or a bug. Need
>> some explanation.
>>
>>
>> My problem is mainly how to run these tests? As unit tests they are
>> pure, does not manipulate any files on disks, does not open socket or
>> anything. (Michael Feathers use the unit test word as strict as this.)
>> So, I expected to run them by anyone, who could compile the sources.
>>
>> #1 'cabal test' could be the best way to exercise them. But the user
>> hook is so complicated for a simple 'compile and run'. Found others
>> ignoring the arguments and call System.Cmd, execute _the compiler_.
>> The compiler name is wired and dependencies also not visible enough,
>> not to mention the extension handling. Basically compile by hand is
>> against cabal! :) (Pro: the best way to invoke; Con: very hard to do)
>>
>> #2 Create testing flag in the cabal file, which enable an executable
>> to be build. (Xmonad guys doing this way.) It compiles easy, developer
>> need not worry about different compilers. Running the test can be done
>> by hand (or darcs) or the test user hook can execute it. If the main
>> product is a library, then it does not link against that, since the
>> library is not installed. Instead developers specify the source dir
>> and it recompiles the tested sources. (Pro: easy to do; Cons: does not
>> test against library)
>>
>> #3 Create another cabal file. Roughly speaking it is same as #2. In my
>> opinion it is suite best for functional tests. (Pro: easy to do; Cons:
>> testing target must be installed first)
>>
>> #4 Use the QuickCheck script. Based naming convention this script runs
>> through the sources, 'grep' the properties, and exec an interpreter to
>> run them. (Pro: easy; Cons: interpreter name is wired in the script)
>>
>> #5 As a proposed long term solution: I might add a new parameter to
>> the 'executable' region describe the target is a unit test. Compile
>> (and run) it only on 'cabal test' invocation. (Con: It does not exists
>> and might be hard to implement :))
>>
>> Thanks for any help!
>>
>> Regards,
>> Laszlo
>>
>> _______________________________________________
>> cabal-devel mailing list
>> cabal-devel at haskell.org
>> http://www.haskell.org/mailman/listinfo/cabal-devel
>>
>