[Haskell-cafe] Re: GSoC: Improving Cabal's Test Support

[Gregory Crosswhite]

Sounds like a good plan to me!

You still need some way to separate out the test dependencies from the main build dependencies, though.  Perhaps as a separate line in the main Cabal header section?

Also, per my earlier comment, it would be good if there were also an "Automatic" option in each test section that defaults to True and specifies whether a test is included automatically in "cabal test" or whether it needs to be specified manually.

Cheers,
Greg


On Apr 6, 2010, at 5:49 PM, Thomas Tuegel wrote:

> I apologize for spamming, but it only just occurred to me how to get
> the best aspects of both our ideas:
> 
> On Tue, Apr 6, 2010 at 8:28 PM, Thomas Tuegel <ttuegel at gmail.com> wrote:
>> On Tue, Apr 6, 2010 at 7:43 PM, Rogan Creswick <creswick at gmail.com> wrote:
>>> The existing Executable sections may serve the need fine, if we could
>>> specify how to run the tests in a different way.  Perhaps a list of
>>> test commands could be specified instead, eg:
>>> 
>>>> TestCommands: foo-test-ghc6.6,
>>>>                            foo-test-ghc6.8,
>>>>                            foo-props --all
>>> 
>>> Anyhow, just food for thought.
>> 
>> One of the reasons I prefer implementing a dedicated 'Test' stanza is
>> that it makes it easier to tell which executables to install.  There
>> may be situations where we want to install test programs, but there
>> will _always_ be situations where we _don't_ want to.  (Maybe
>> '--{en,dis}able-tests' passed to 'cabal install' should control this?)
>>  Could we take a 'TestCommands' list and parse out the options to get
>> the executable names?  Yes, but relying on that to work makes me
>> uneasy.  I think the more robust way to specify executable-specific
>> options instead would be to add a field to the 'Test' section
>> ('run-options' seems to be consistent with the naming scheme) that
>> doesn't exist in 'Executable'.
> 
> Suppose we adopt your suggestion and let test programs be ordinary
> executables in ordinary 'Executable' sections, and make 'Test'
> sections that look like:
> 
>> Test foo-1
>>    exe-is: foo
>>    options: --enable-bar --disable-baz
>> 
>> Test foo-2
>>    exe-is: foo
>>    options: --enable-bar --enable-baz
>> 
>> Test some-other-test
>>    exe-is: some-other-test
>>    options: --something-completely-different
> 
> Now we get named sets of options but don't have to worry about parsing
> through to find the executable names!  It also solves the problem of
> running test suites with multiple sets of options at runtime.
> 
> -- 
> Thomas Tuegel
> _______________________________________________
> Haskell-Cafe mailing list
> Haskell-Cafe at haskell.org
> http://www.haskell.org/mailman/listinfo/haskell-cafe