patch: --enable-tests and --only-dependencies

[Thomas Tuegel]

On Fri, Feb 3, 2012 at 4:40 AM, Andres Löh <andres at well-typed.com> wrote:
>> The way to handle tests for named packages would be to add a
>> PackageConstraint for tests the way is already done for flags, but
>> this is a lot more work because it would require getting deep into the
>> dependency solver. However, it would make it easy to set the config
>> flags during build for explicit targets so we could implement a test
>> auto-run feature.
>
> When I rewrote the solver, I was actually surprised that test flags
> are sort of bypassing the solver. I don't think it would be all that
> difficult to build it into the new solver properly, *if* I'd know what
> properly means. So if you can specify a reasonable behaviour you'd
> expect from the solver, I can try to implement it.

I looked at the top-down solver again to see if we could enable/disable tests
and benchmarks through a constraint (the way flag assignments are done). To my
surprise, it's not nearly as confusing as I thought it was the last time I
looked at it (two years ago (!) during GSoC when I wrote that awful
solver-bypassing test code).

The attached patches do what I think we want. Test dependencies for targetted
packages (named and source alike) are detected and installed, but
'--enable-tests' is not propagated to dependencies. I only checked the top-down
solver because I'm still not familiar with the modular solver. I also included a
patch to automatically run the test suites when 'cabal install --enable-tests'
is invoked and abort installation if the tests fail. I think this is a sensible
default; if a user is going to ignore the results of the test suites, they
probably don't want '--enable-tests'.

Johan and Antoine: I tested my patch with some source packages I have around and
the packages I could find on Hackage that have test suites, but that only
assures me that '--enable-tests' does what _I_ think it should do. It sounds to
me like you use this feature on a daily basis: could you try out this set of
patches and let me know if this behavior meets your needs?

Andres: I had to make a few changes to the modular solver to get
cabal-install to
compile with the new constraints, but I didn't go far enough to actually have
the solver pull in the dependencies. I recorded these changes as a separate
patch to make the changes more obvious. You understand the solvers better than I
do; if I'm taking the wrong approach, please let me know. Otherwise, this is
what I did outside the solvers, if you want to implement test and benchmark
dependencies for the modular solver:

1. There is a new type, 'OptionalStanza', with two constructors indicating
respectively that '--enable-tests' or '--enable-benchmarks' was specified.
2. Constraints are passed to the solver in the same way as flag assignments. The
names of the target packages are determined, and a list [OptionalStanza] is
associated with each. If a constructor is present in the list, that type of
stanza is enabled. If the list is empty, neither tests nor benchmarks are
enabled.

When a solver is building an InstallPlan, it just needs to configure each
package respecting the constraints to get the correct list of dependencies.
Packages that aren't specified as targets will have no constraints, so they
should be built without '--enable-tests' and '--enable-benchmarks'.

Hopefully we can put this long-standing bug to rest!

Thanks!

-- 
Thomas Tuegel
-------------- next part --------------
A non-text attachment was scrubbed...
Name: cabal-install-enable-tests.dpatch
Type: application/octet-stream
Size: 69263 bytes
Desc: not available
URL: <http://www.haskell.org/pipermail/cabal-devel/attachments/20120207/b8762408/attachment-0001.obj>