Proposal: Add a benchmark section to .cabal files

[Duncan Coutts]

On Wed, 2011-10-05 at 12:09 +0200, Tillmann Rendel wrote:
> Hi,
> 
> Johan Tibell wrote:
> > [...] I'm convinced we need support for building benchmark using Cabal.  [...]
> 
> Could test sections be used for benchmarks, too?
> 
> > Step 1. Add basic support for building simple benchmark suites e.g. modelled
> > after test suite sections:
> 
> This should work immediately.
> 
> > Step 2. Add a detailed benchmark interface that allows the benchmark to
> > associate a real valued *measure* (optionally with upper and lower bounds)
> > with a benchmark name.
> 
> This could be supported by a protocol where a "test" reports a real 
> instead of a boolean value.
> 
>    Test bench-foo
>        type:           benchmark-1.0
>        main-is:        bench-foo.hs
>        build-depends:  base
> 
> Except for the more specialized keywords, there seems to be no need to 
> distinguish tests from benchmarks.

Hmm, that's an interesting idea, to just make benchmarks a different
testsuite protocol. Certainly we could prototype it like this.

Duncan