new i/o library

[Einar Karttunen]

On 27.01 13:10, Bulat Ziganshin wrote:
> i'm now write some sort of new i/o library. one area where i currently
> lacks in comparision to the existing Handles implementation in GHC, is
> the asynchronous i/o operations. can you please briefly describe how
> this is done in GHC and partially - why the multiple buffers are used?

One simple optimization is that you can omit all buffering with
unbuffered operation. Then simply add the buffer (which is ok
because Handles are mutable) if the user ever calls hLookAhead.

> moreover, i have an idea how to implement async i/o without complex
> burecreacy: use mmapped files, may be together with miltiple buffers.
> for example, we can allocate four 16kb buffers. when one buffer is
> filled with written data, the program unmaps it and switches to use
> the next buffer. i don't tested it, but OS can guess that unmapped
> buffer now should be asynchronously written to disk. the same for
> reading - when we completely read one buffer, we can unmap it, switch
> to the second buffer and map the third so that the OS can
> asynchronously fill the third buffer while we are reading second.
> should this work, at least on the main desktop OSes?

Please no. There are multiple reasons to avoid mmapped files.
1) They make very few performance guarantees for reading
   (i.e. a Haskell thread touches memory which has not yet
    been read from the file causing IO and all the other
    Haskell threads are blocked too)
2) The time of writes is unpredictable making implementing a
   hFlush harder? (not sure about this)
3) Not all file descriptors will support it - i.e. we will
   need the read/write path in any case.
4) Mmap cannot be used for random access for arbitrary files
   since they may be larger than the address space. This means
   some kind of window needs to be implemented - and this is
   easily done with read/write.

- Einar Karttunen