[Haskell-cafe] Re: Emulating bash pipe/ process lib

[Simon Marlow]

Bulat Ziganshin wrote:
> Hello Donn,
> 
> Friday, February 10, 2006, 12:47:42 AM, you wrote:
> 
> 
>>>DC> "Slow" devices like pipes, sockets etc. get along fine with Handles
>>>DC> or whatever buffered I/O - as long as you have only one going at a time.
>>>DC> Multiple input sources - like, say you want to read a process' output
>>>DC> (unit 1) and diagnostic output (unit 2) separately, and either one has
>>>DC> the potential to fill up the pipe and block while you're waiting for
>>>DC> input on the other pipe - buffers at least complicate the dispatching
>>>DC> mechanics you need for this, if not make it impossible.
>>>
>>>are you tried LineBuffering and NoBuffering? seem that it is developed
>>>exactly for this case (plus for terminals)
> 
> 
> DC> That's part of the idea, it helps keep the data out of buffers where
> DC> select or whatever can't see it.
> 
> DC> But then you need functions with semantics that really support unbuffered
> DC> I/O.  When select tells you that the device is readable, you don't know
> DC> that there is one full line, or how many bytes there are, so hGetLine
> DC> doesn't apply here, nor would the Haskell equivalent of fread(3) if there
> DC> were one.  The only thing left is hGetChar - one char, then select, then
> DC> another.  (And multi-byte chars cause us to worry about this.)
> 
> when i think how to implementat LineBuffering, i decided that it is
> the only possible way - read byte a time and see for a '\n'. i don't
> know how System.IO implemented but i think that it should do the same

Read as much as you can into the buffer without blocking.  Line 
buffering on input is actually implemented exactly the same as block 
buffering.

You might argue that strictly speaking this isn't line buffering, since 
you can get data from the Handle before the end of line is available. 
That's true, but I'd argue this is more useful.  In fact, we changed 
block buffering on input handles so that the input buffer doesn't have 
to be completely full before data can be returned, which is also not 
strict block buffering, but seems more useful.

I suppose conceivably you might want to force a read buffer to be 
completely full so that you could guarantee to read it all without 
blocking, but in that case you might as well use hGetBuf & peekArray. 
Similarly you might want to ensure the buffer contains a complete line 
before starting to read it, but can use hGetLine anyway.

Cheers,
	Simon