[Haskell-cafe] Problems using tasty-golden

[Jan Stolarek]

Hi *,

I've been trying to write a testsuite using tasty-golden and I've run into two technical problems. 

I'll start with some background. I'm working on singletons library that provides Template Haskell 
functionality. I would like to tests whether some particular TH splices compile at all and if 
they compile whether they produce expected output. My plan is to attempt compilation of a file 
with GHC. If compilation fails then the test fails. If it succeedes then output is compared with 
a golden file. I am directly calling goldenTest function from Test.Tasty.Golden.Internal. I wrote 
this function to produce the test output:

compileWithGHC :: IO (Either BSL.ByteString BSL.ByteString)

It runs GHC with some specified options and returns Left stderr (if compilation fails) or Right 
stdout (if it succeedes). This function is passed as third parameter to goldenTest (using 
liftIO). Then I have a comparing function:

cmp :: Either BSL.ByteString BSL.ByteString 
     -> Either BSL.ByteString BSL.ByteString 
     -> IO (Maybe String)

If it gets two Rights then it compares them. If the second param is left then the function returns 
Just stderr.

I've run into following problems:

1. When I try to run my test I get:

message threw an exception: MY_GOLDEN_FILE: hGetBufSome: illegal operation (handle is closed)

I suspect that lazy IO is getting in the way but I have no idea why exactly things go wrong.

2. I had to hardcode the path to my golden file. Otherwise the file could not be found. I suspect 
the reason is that test are run somewhere in dist/ subdirectory whereas my golden files are 
located alongside my test files. What can I do to have the golden files located without giving 
specific subdirectories in the source code?

A bonus question. I redefined vgReadFile from Test.Tasty.Golden.Internal to return Either:

vgReadFileE :: FilePath -> ValueGetter r (Either BSL.ByteString BSL.ByteString)
vgReadFileE path =
  (fmap Right . liftIO . BSL.hGetContents =<<) $
  ValueGetter $
  ContT $ \k ->
  bracket
    (openBinaryFile path ReadMode)
    hClose
    k

The only modification is the "fmap Right .". How can I return Left if opening the file fails?

Janek