ByteString I/O Performance

[Duncan Coutts]

On Tue, 2007-09-04 at 20:07 +0200, Peter Simons wrote:
> Donald Bruce Stewart writes:
> 
>  >> hGet :: Handle -> ByteString -> IO ByteString
>  >> hGet h buf = do i <- Str.unsafeUseAsCStringLen buf (\(p,n) -> hGetBuf h p n)
>  >>                 return (Str.unsafeTake i buf)
>  >
>  > That's a useful benchmark. Thanks for looking into this.
> 
> It was a pleasure. How do you feel about providing this kind of
> input API to the user as part of the module?

We can't provide this interface as it modifies the immutable input
ByteString.

What you're looking for is to copy into a mutable buffer. That buffer
cannot be itself a ByteString as they're immutable. The best we can do
is to copy into a newly allocated buffer as we do in hGet. In the best
case we can do this with a single copy. We could look at the trimming
again if that's an issue.

So if you want an api that provides a mutable input buffer, you'll need
something other than a ByteString. We do have something like this in the
binary package, which we've been considering cleaning up and bringing
into the bytestring package. However even that doesn't give you a
mutable buffer that you'd want for fast 'cat', that really requires an
api that guarantees that the buffer is no longer in use when the next
chunk is read, destroying the previous content of the buffer.

> By the way, did you receive my e-mail about the race condition in
> readFile?

Yes, it's a good point. We could fix it my doing more reads in the same
style a hGetContents.

Duncan