[Haskell-cafe] Re: GSoC: Improving Cabal's Test Support

[Rogan Creswick]

On Wed, Apr 28, 2010 at 1:24 AM, Richard G. <richardg at richardg.name> wrote:
> I think that formatted plain-text output would be much better than XML,
> something that is human-readable and relatively easy to parse via machine.
>  Something similar to the GHC error output would work well because
> developers are familiar with it.

I don't think we need to be limited to a single output format.  It's a
simple thing to have continuous integration (or cabal) invoke tests
with a flag/option to output in a specific format.  XML is useful
because there are a number of mature tools that already expect xml --
we don't need to reinvent the wheel to get some of the capabilities
that  developers in other languages are enjoying if our tools use some
of the same formats (despite the issues that may exist with those
formats..).

I like your suggestion for an emacs/dev-readable format, and it can
coexist with xml and other "snazzier" outputs (such as the default
format for test-framework, which uses many little tricks to draw and
erase progress bars / etc.)

--Rogan

>
> Test <n>:<Result>
>    <Location>
>    <Error message>
>
> E.g.,
>
> Test 1:Passed
>    src/Some/File.hs:23
>
> Test 2:Failed
>    src/Some/File.hs:27
>    Expecting `4'; received `5'.
>
> Test 3:Error
>    src/Some/OtherFile.hs:39
>    Unexpected exception.
>
> This would keep the complexity low in Cabal and allow for easy
> transformation to XML.
>
> Richard G.
>
> On 10-04-08 8:30 PM, Rogan Creswick wrote:
>>
>> On Thu, Apr 8, 2010 at 5:53 AM, Duncan Coutts
>> <duncan.coutts at googlemail.com>  wrote:
>>>
>>> I think it's important to be able to convert into standard or custom
>>> formats. I've no idea if JUnit XML would make sense as the native
>>> format. It's plausible.
>>>
>>
>> I hadn't really thought about cabal, itself, being a consumer for test
>> results -- but I like your (Duncan's) points about defining a testing
>> interface, and keeping it extensible.
>>
>> For the record: I don't think junit xml is a good choice for a native
>> format :), but I do think it's a good format to start with simply
>> because there are many tools that can consume it already.
>>
>> --Rogan
>>
>>
>>> Duncan
>>>
>>> _______________________________________________
>>> Haskell-Cafe mailing list
>>> Haskell-Cafe at haskell.org
>>> http://www.haskell.org/mailman/listinfo/haskell-cafe
>>>
>> _______________________________________________
>> Haskell-Cafe mailing list
>> Haskell-Cafe at haskell.org
>> http://www.haskell.org/mailman/listinfo/haskell-cafe
>>
> _______________________________________________
> Haskell-Cafe mailing list
> Haskell-Cafe at haskell.org
> http://www.haskell.org/mailman/listinfo/haskell-cafe
>