[Haskell-cafe] runInteractiveCommand: program ends before writing or reading all the output

[Duncan Coutts]

On Thu, 2008-05-15 at 13:40 -0400, Olivier Boudry wrote:
> Hi all,
> 
> It's the first time I use the runInteractiveCommand and I was probably
> bitten by laziness.

Yes. I think Philip diagnosed the problem correctly.

As an example let me show you as an example how we use it in Cabal:

rawSystemStdout :: Verbosity -> FilePath -> [String] -> IO String
rawSystemStdout verbosity path args = do
  Exception.bracket
     (runInteractiveProcess path args Nothing Nothing)
     (\(inh,outh,errh,_) -> hClose inh >> hClose outh >> hClose errh)
    $ \(_,outh,errh,pid) -> do

      -- We want to process the output as text.
      hSetBinaryMode outh False

      -- fork off a thread to pull on (and discard) the stderr
      -- so if the process writes to stderr we do not block.
      -- NB. do the hGetContents synchronously, otherwise the outer
      -- bracket can exit before this thread has run, and hGetContents
      -- will fail.
      err <- hGetContents errh 
      forkIO $ do evaluate (length err); return ()

      -- wait for all the output
      output <- hGetContents outh
      evaluate (length output)

      -- wait for the program to terminate
      exitcode <- waitForProcess pid

      -- on failure, throw the exit code as an exception      
      unless (exitCode == ExitSuccess) $ exitWith exitCode

      return (output, exitcode)


So as you can see there are two subtleties. One is the issue that we
have to make sure we get all the output before we wait for the program
to finish. Using evaluate is the trick there.

The other is that we also have to pull on the stderr of the process.
Otherwise the process may be trying to output to stderr but if the pipe
buffer fills up then writing to stderr will block and so it will not be
able to continue writing to stdout we'll have deadlocked the process. So
we have to forkIO a thread to pull on stderr.

This is one reason that runInteractiveProcess is hard to use and
especially hard to use portably due to the use of preemptable threads.
It would be possible to do without threads if we used non-blocking IO
and interleaved between reading from stdout and stderr.

Duncan