working on a fix for ticket #212

[Laszlo Nagy]

Hi All,

I updated my patches related to the test suite.

  http://sites.google.com/site/rizsotto/patches/cabal-ticket212.darcs
  http://sites.google.com/site/rizsotto/patches/cabal-ticket484.darcs
  http://sites.google.com/site/rizsotto/patches/cabal-system.platform-fix.darcs

Regards,
Laszlo


On Sun, Jun 14, 2009 at 9:33 PM, Duncan
Coutts<duncan.coutts at worc.ox.ac.uk> wrote:
> On Wed, 2009-05-13 at 22:32 +0200, Laszlo Nagy wrote:
>> Hi All,
>>
>> Trying to create unit test suite.. work in progress. Comments are more
>> than welcome!
>>
>> http://sites.google.com/site/rizsotto/patches/cabal-ticket-212-v1.darcs-send
>
> Hi Laszlo,
>
> I'd like to get your tests integrated, in addition to the ones Stephen
> added recently. We can sort out the framework to use after getting them
> in. I expect Stephen is right that in the end we should use
> test-framework so that we can handle QC and HUnit based tests. It also
> looks like a reasonable framework generally.
>
> However the first thing to do is to get your existing tests in. Can I
> ask you to make a couple minor organisational changes?
>
> Instead of one massive tests/Test/Properties.hs file containing all the
> tests for the Distribution/* modules, could we use one module of
> properties for each module we're testing?
>
> So for example you integrated the existing
> tests/Test/Distribution/Version.hs
> into your
> tests/Test/Properties.hs
> but perhaps we could do the reverse and split it out into
> tests/Test/Distribution/License.hs
> tests/Test/Distribution/Package.hs
> tests/Test/Distribution/Compiler.hs
> tests/Test/Distribution/System.hs
> tests/Test/Distribution/ModuleName.hs
> tests/Test/Distribution/PackageDescription.hs
>
> That would make it much easier for someone to go and find the properties
> they're looking for. I realise that many tests will share the same
> utility functions and generators etc. That's fine. For example the
> Test.Distribution.System module would define the generator for the OS
> and Arch types which are needed eg in
> Test.Distribution.PackageDescription. So just import one in the other.
> For things which do not belong to a specific module, we can just use
> shared utility modules like the existing
> tests/Test/QuickCheck/Utils.hs
> or any other name that seems appropriate.
>
>> I wrote simple properties like this, for all data type
>>
>> > prop_parse x = x == parse $ display x
>>
>> I did merged the existing QuickCheck test cases into one file. I found
>> it easier to reuse the generator functions for different types.
>>
>> My findings:
>>  - The System.Platform did not passed parse tests. The reason: both
>> System.OS and System.Arch allowed the dash character in the name. But
>> System.Platform text representation was concatenate the two strings
>> separated by a dash. So I just simple removed the dash from them.
>> Solved! ;)
>
> Great. Could that be made into a separate patch from the addition of the
> tests? The commit message can say exactly what you just said in
> explanation of the change.
>
>> - VersionRange does not passes parse tests, iff more than two ranges
>> is present. As far as I could see, the parse method is wired to handle
>> only two ranges. Not corrected, because found no good solution. Need
>> help on this!
>
> Right, currently the parser only allows one conjunction and no brackets.
> There's an open ticket to allow more general expressions.
> http://hackage.haskell.org/trac/hackage/ticket/484
> For the moment then it's ok for this test to fail. We can expect it'll
> be fixed when ticket #484 is completed.
>
>> - Package.Dependency also not passes parse tests. (It is depend on
>> VersionRange success.)
>
> Right.
>
>> - Compat.ReadR gather property is obsolete and does not compile. Can't
>> figure it out how could I rewrite it. Need help!
>
> Easy solution! The gather function is not used. We can just remove
> it :-)
>
>> - PackageDescription does not passes simple parse test. I guess the
>> problem is with the test itself! ;) I used showPackageDescription
>> method to generate string representation of this type and parse it
>> back. I could not judge it is an expected behaviour or a bug. Need
>> some explanation.
>
> The showPackageDescription is totally borked. It really doesn't work at
> all. Again it's ok for the moment if this test just fails. I'll be
> motivation for someone to fix showPackageDescription.
>
>> My problem is mainly how to run these tests?
>
> Lets get the tests in first and then sort this out.
>
>> As unit tests they are pure, does not manipulate any files on disks,
>> does not open socket or anything. (Michael Feathers use the unit test
>> word as strict as this.) So, I expected to run them by anyone, who
>> could compile the sources.
>
> Yes, pure tests are preferable wherever possible.
>
>> #1 'cabal test' could be the best way to exercise them. But the user
>> hook is so complicated for a simple 'compile and run'. Found others
>> ignoring the arguments and call System.Cmd, execute _the compiler_.
>> The compiler name is wired and dependencies also not visible enough,
>> not to mention the extension handling. Basically compile by hand is
>> against cabal! :) (Pro: the best way to invoke; Con: very hard to do)
>
> Yeah the test hook is pretty much useless. We've got a design for an
> improved test system but it may be a while before we implement it.
>
> So that would be the long term solution, and where better to try it out
> that Cabal's own tests. In the mean time we can sort out something, even
> if it's a little hacky. I can do that bit.
>
> Duncan
>
>