I/O overhead in opening and writing files

[wren ng thornton]

On 8/27/12 5:33 PM, Donn Cave wrote:
> It does, but it can also read directly to Ptr Word8 (fdReadBuf), which
> you'd think would be closer to hardware speed - but then you might lose
> the advantage trying to peek the data out of the buffer.  In principle
> you ought to be able to stuff that pointer right into a ByteString,
> but don't know for sure that there's any public API for such.

As Austin Seipp mentioned, there's unix-bytestring[1] which minimizes 
the amount of marshaling/conversion imposed by using a high-level 
language. And it includes the obvious conversion between Ptr Word8 and 
ByteString. If there's any remaining overhead, let me know and I'll do 
my best to eliminate it.

But, that's only for the reading and writing; opening files is another 
matter. If it's the *opening* of files that's causing the slowdown, then 
that has to be due to something in how GHC handles filename conversion 
et al.


[1] http://hackage.haskell.org/package/unix-bytestring

-- 
Live well,
~wren