Cabal's detailed test interface

[Thomas Tuegel]

On Wed, Jul 27, 2011 at 5:06 AM, Johan Tibell <johan.tibell at gmail.com> wrote:
> A few notes:
>
> * Calling the data type that contains the Test, Group, and
> ExtraOptions constructor Tests feels a bit weird, as you often end up
> having to give a list of these e.g. to the Group constructor. You end
> up with something of type [Tests]. I think Test is a better name for
> this type.

It's true; I _do_ feel weird having something with the type [Tests]. I
will change it.

> * Having the top-level 'tests' binding be IO Tests, instead of IO
> [Tests], is a bit inconvenient. It forces the users to always use at
> least one group, which forces him/her to name that group. As we all
> known, naming things is one of the two hard problems in computer
> science (the other being cache coherency).

If we do this, should test runners assume that the list of tests can
safely be run concurrently, or not? My primary motivation for
requiring the user to use a group was the explicit answer to this
question.

> * I'm a bit confused about how Progress would be used in practice. It
> returns a String. What is the client supposed to do with this string?
> Returning a String doesn't seem enough to implement some kind of
> update-in-place cursors interface.

The String is intended to be a status message, something to display to
the user. For comparison, the intermediate types used by
test-framework are only required to have a Show instance, so I think
this is sufficient.

> * Perhaps we should expose some convince functions to create Test
> values. In particular I feel like you'd rarely want to create a
> TestInstance value without wrapping it in a Test constructor. I like
> the testGroup function exported by test-framework.

I will have a look at the convenience functions test-framework offers,
and see which would be useful here.

Thanks!

-- 
Thomas Tuegel