popen bug?

[David Roundy]

On Mon, May 05, 2003 at 11:10:51AM +0100, Mark Preston wrote:
> Is there a bug in the GHC implementation of the popen function that 
> manifests itself when more than 4k is written to the pipe?
> 
> I'm trying to reproduce in Haskell the standard UNIX shell pipe idiom:
> 
>    cat 4097 | cat
> 
> where 4097 is a file containing 4097 bytes.

I'd say that popen is trying to write too much data to the pipe and running
into the pipe limit.  Really it should create a separate thread for writing
to the pipe (which is what I do...), so that it can feed the pipe a little
data at a time.  The problem is that while a pipe looks like a normal file,
it is one to which a write may block, or if you use nonblocking IO, the
write may need to keep trying... so I'd say that yes, it is a bug in popen,
but a rather tricky one to fix in a portable manner.  I suppose in the
posix library one can assume that one is in a posix system and comfortably
use fork on the C side, which is what I do.

I only know about this because I spent all Saturday morning struggling with
this very issue, except that I was using ffi and C to communicate with the
pipes.  If you're interested in my solution (which is an embarrassingly
ugly one), I'd be happy to share it with you.  I guess you could also look
a the code at the following:

http://www.abridgegame.org/cgi-bin/darcs?darcs**

See the patch whose name starts with "ugly code", with the relevant code
being in the inappropriately chosen files Curl.hs and hscurl.c.  In
particular, you could look at the haskell function runPipe, which I think
does basically what you are trying to do.
-- 
David Roundy
http://www.abridgegame.org/darcs