[Haskell-cafe] QuickCheck - Extracting data values from tests

[Jorge Adriano Aires]

> > Hello all,
> > When using Quickcheck, is there some way to extract generated data values
> > to the IO Monad?
> >
> > I know I can collect and print information about test cases, but that's
> > not enough. Data may be pretty complex, and there may be no parsers for
> > it. If a test suddenly goes wrong, just having it displayed doesn't seem
> > that useful.
>
> You may be interested in a QuickCheck hack of mine that saves the offending
> data value to use immediately in the next test run.

Nice!
It's different from what I was looking for but also quite usefull.

> You can get the current version with
> "darcs get http://thunderbird.scannedinavian.com/repos/quickcheck"
> I've only used this for my own code, so I'd be interested in any feedback.

Ok.

> In some cases it's a lot easier to generate a value from a seed and size
> rather than saving the value in some way that you can restore (ie
> functions).
>
> I've been investigating doing test-driven-development with QuickCheck,
> saving failing test cases is one step towards that goal. If you have more
> ideas on that topic, I'd like to hear about it.

Well, returning (part of) the generated data is one of them :)

> > Also, even when I'm implementing a generator, I want to see how it is
> > working. Running a verboseCheck on some dummy property helps, but I may
> > want to analyse the data, or some parts of it better - for instance, for
> > many data structures I have alternative "show" functions that take
> > parameters as arguments.
>
> This isn't clear to me, can you give other examples?


Not sure which part is not clear... I'll just try to explain each of them.
Lets say I'm implementing a generators for Graphs.

> > Also, even when I'm implementing a generator, I want to see how it is
> > working.
I want to check if the generated Graphs are like I intended them to be.

> > Running a verboseCheck on some dummy property helps, but I may
verboseCheck by default prints all the data.
I can run it on a dummy function that always returns True to see what kind of 
data I'm getting.


> > want to analyse the data, or some parts of it better
May want to print the 'actual graphs' on the screen (ASCII art, or maybe using 
some function that calls Gnuplot). Then I may want to check in more detail 
the info in contained in some of the nodes. Then I may decide to run some 
functions on it.

> > many data structures I have alternative "show" functions that take
> > parameters as arguments.
Like I just said, I may want to "show" the graph in many ways.


But there are more possibilities.
Why limitate the usefulness of QuickCheck? Suppose I just implemented 
generators for a few kinds of terms and formulas to test some properties. Now 
I want to benchmark a couple of different unification functions... I'd expect 
to be able to use my generator for that. Unless I'm missing something, I 
cannot. Am I right?

J.A.