[Haskell-cafe] ANNOUNCE: test-framework-golden-1.1

[Roman Cheplyaka]

* Simon Hengel <sol at typeful.net> [2012-10-07 15:45:21+0200]
> On Fri, Oct 05, 2012 at 05:17:18PM +0300, Roman Cheplyaka wrote:
> > I can do that indeed, and I guess I could reimplement everything I have
> > at the moment on top of HUnit.
> > 
> > However, an important part of functionality isn't there at the moment —
> > golden file management. You should be able to say, "for this test,
> > take its current output and write it to the corresponding golden file".
> > 
> > In order to do that, you need to have access to the list of golden tests
> > in the suite. This is where implementation details of different test
> > frameworks start to matter. Probably one can make an abstraction over
> > test frameworks that would give the list of all golden tests.
> > (Although when you start abstracting over test frameworks, which are
> > abstractions themselves, it becomes somewhat funny.)
> 
> Ok, makes sense.
> 
> I'm looking forward to give it a try, and see how it compares to using
> operating system primitives (say `cp') for "golden file management".

1. You often want to update not just one test, but all, or some of the
   tests (when you've made a change and verified that the changes in
   output are expected). Doing it in command line is certainly possible,
   but not trivial nor convenient.
2. For some tests (like goldenVsString) the output is not captured in a
   file, so using cp directly is not possible.

> > Speaking of such functionality, correct me if I'm wrong, but neither
> > HUnit nor hspec won't be able to support it anyway, because they
> > represent tests as opaque IO actions.
> 
> It would be easy to extend Hspec to support this in the same way you
> extend test-framework to support this.  It requires existentials;  the
> only substantial difference that I can see is that test-framework
> already uses existentials, while Hspec does not.

Well, if you are willing to make this change, then I'll try to do my
part of the job and expose a useful abstraction.

Roman