Cabal's detailed test interface

[Johan Tibell]

Curious about how a user of this API would look like I converted the
parts of the test-framework API I use (for QC2 tests) to use the Cabal
API.

    -- Converting a QC2 property to a Cabal test type.

    type TestName = String

    testProperty :: Testable a => TestName -> a -> Tests
    testProperty prop name = Test $ TestInstance {
          run = quickCheck ...
        , name = name
        , tags = []
        , options = []
        , setOption = \ _opt _val -> Left "No such option"
        }

    -- A lists of 'Tests' feels a bit unnatural, as "Tests" is already
    -- plural.
    testGroup :: TestName -> [Tests] -> Tests
    testGroup name = Group name True

    -- The actual tests

    prop_reverse :: Testable a
    prop_sort :: Testable a

    -- Always having to use a group feels a bit cumbersome (coming up with
    -- names for things is hard!).  Perhaps the top-level export should be
    -- a list?
    tests = testGroup "Data.List"
            [ testProperty "reverse/reverse" prop_reverse
            , testProperty "sort" prop_sort
            ]

A few notes:

* Calling the data type that contains the Test, Group, and
ExtraOptions constructor Tests feels a bit weird, as you often end up
having to give a list of these e.g. to the Group constructor. You end
up with something of type [Tests]. I think Test is a better name for
this type.

* Having the top-level 'tests' binding be IO Tests, instead of IO
[Tests], is a bit inconvenient. It forces the users to always use at
least one group, which forces him/her to name that group. As we all
known, naming things is one of the two hard problems in computer
science (the other being cache coherency).

* I'm a bit confused about how Progress would be used in practice. It
returns a String. What is the client supposed to do with this string?
Returning a String doesn't seem enough to implement some kind of
update-in-place cursors interface.

* Perhaps we should expose some convince functions to create Test
values. In particular I feel like you'd rarely want to create a
TestInstance value without wrapping it in a Test constructor. I like
the testGroup function exported by test-framework.

Cheers,
Johan