Cabal's detailed test interface

[Thomas Tuegel]

On Mon, Aug 8, 2011 at 5:52 PM, Duncan Coutts
<duncan.coutts at googlemail.com> wrote:
> On Thu, 2011-07-28 at 21:25 -0500, Thomas Tuegel wrote:
>> I can see the utility of this. It would be simple enough to have Group
>> take an (IO [Tests]) instead of just [Tests].
>
> Note that that's not quite the same thing. That lets you do IO to
> enumerate the tests, not IO before and after running a group of tests.

> I'm slightly confused about what you and Johan are talking about with
> [Tests] etc. Can you post what the new proposed data types are please?

Johan and I are talking about renaming the Tests type to just "Test"
to be more consistent about pluralization. So, also incorporating the
changes I think you are suggesting to the Group type, it would look
like:

data Test
    = Test TestInstance
    | Group
        { groupName     :: String
        , concurrently  :: Bool
        , groupTests    :: [Test]
        , preGroup      :: IO ()
        , postGroup     :: IO ()
        }
    | ExtraOptions [OptionDescr] Test

Test modules would export:

tests :: IO [Test]

> As for convenience functions, I don't know if it's worth it. We're not
> expecting packages to implement this directly. It'll just be for
> packages like test-framework to implement. But if they're cheap and
> obvious then I don't object strongly.

The most important one I can think of would be

testGroup :: String -> [Test] -> Test    -- Make a Group with
concurrently = True and no preGroup or postGroup

I would expect test frameworks to provide interfaces that create a
Test directly, rather than returning a TestInstance. I think that and
the above function would cover all the common use cases.

-- 
Thomas Tuegel