[Haskell-cafe] HUnit false-positive stumper
KQ
quick at sparq.org
Mon Jun 6 03:34:54 CEST 2011
While working on a project of mine recently I realized that a particular HUnit test should have been failing. After paring things down, I came up with this shocker:
test_perhaps.hs:
> module Main where
>import Test.HUnit
>main = runTestTT $ TestList [ True ~=? True
> , False ~=? True
> , TestCase $ assertEqual "both true" True True
> , TestCase $ assertEqual "false true" False True
> ]
$ cabal-dev install
...
$ ./cabal-dev/bin/test_perhaps.hs
### Failure in: 3
false true
expected: False
but got: True
Cases: 4 Tried: 4 Errors: 0 Failures: 1
The shock here is that there was only one failure, whereas the "False ~=? True" should have failed.
Environment: MacOS 10.5.8 (Leopard)
GHC 6.12.3
cabal-dev 0.7.4.1
Cabal 1.10.1.0
HUnit 1.2.2.3
I even tried removing the dist and cabal-dev directories and reattempting but got the same results.
I tried the same test in a different environment (Linux PC w/GHC 7.0.2, all other elements the same) and got correct results:
$ ./cabal-dev/bin/test_perhaps
### Failure in: 1
expected: False
but got: True
### Failure in: 3
false true
expected: False
but got: True
Cases: 4 Tried: 4 Errors: 0 Failures: 2
$
I realize it's very difficult to debug someone's setup remotely, but I have to confess I'm really stumped at this point. The input program is pretty straightforward, and I *have* gotten test failures in the past on the Mac. I did do a "cabal update" the other day after being informed that my hackage file was getting old, but HUnit hasn't changed in months.
I even tried unpacking HUnit and reducing the (non-)failing test case by using the definitions of the elements:
> module Main where
>
> import Control.Monad (unless)
> import Test.HUnit
>
> main = runTestTT $ TestList [ True ~=? True
> , False ~=? True
> , TestCase $ assertEqual "both true" True True
> , TestCase $ assertEqual "false true" False True
> , TestCase $ assertEqual "fa" False True
> , TestCase $ assertEqual "f" False True
> , TestCase $ (False @?= True)
> , TestCase $ unless (False == True) (assertFailure "f")
> ]
The results are very strange:
$ ./cabal-dev/bin/test_perhaps
### Failure in: 3
false true
expected: False
but got: True
### Failure in: 4
fa
expected: False
but got: True
### Failure in: 7
f
Cases: 8 Tried: 8 Errors: 0 Failures: 3
$
The assertEqual form doesn't fail as it should if the label is a single character (test 5), but in its fully expanded form (test 7) it will fail. Huh?!
At this point I'm thoroughly confused. I'm using 6.12.3 on the Mac because I don't have the newer MacOS release for which there's a Haskell Platform release and I haven't wanted to build GHC by hand. I wouldn't expect 6.12.3 to have issues like this, but I wouldn't expect issues like this anywhere. If anyone has any suggestions (other than upgrading to GHC 7.x) I'll be most appreciative.
--
-KQ
More information about the Haskell-Cafe
mailing list