new i/o library

[Bulat Ziganshin]

Hello Duncan,

Friday, January 27, 2006, 4:00:28 PM, you wrote:

>> moreover, i have an idea how to implement async i/o without complex
>> burecreacy: use mmapped files, may be together with miltiple buffers.
>> for example, we can allocate four 16kb buffers. when one buffer is
>> filled with written data, the program unmaps it and switches to use
>> the next buffer. i don't tested it, but OS can guess that unmapped
>> buffer now should be asynchronously written to disk. the same for
>> reading - when we completely read one buffer, we can unmap it, switch
>> to the second buffer and map the third so that the OS can
>> asynchronously fill the third buffer while we are reading second.
>> should this work, at least on the main desktop OSes?

DC> On Linux an probably other unix-like OSes I don't think this would be
DC> any different from using read/write.

DC> On Linux, read and mmap use the same underlying mechanism - the page
DC> cache. The only difference is that with mmap you get zero-copy access to
DC> the page cache. However frequent mapping and unmapping may eliminate
DC> that advantage. Either way there is no difference in how asynchronous
DC> the operations are.

yes, i want to save exactly this bit of performance - after i
optimized all other expenses on the path of text i/o

in other words, i interested in having zero-wait operation both for
reading and writing, i.e. that in sequence of getChar or putChar
actions there were no waits on any action - of course, if the disk is
fast enough. in other words, speed of such i/o programs should be the
same as if we just write these data to memory

current GHC's Handle implementation uses rather complex machinery for
async reading and writing, and i can even say that most part of
Handle's implementation complexity is due to this async machinery. so
i wanna know what exactly accomplished by this implementation and can
we implement async operation much easier by using mmap?

the word "async" is overloaded here - i'm most interested in having
zero-overhead in single-threaded operation, while GHC's optimization,
afair, is more about overlapping I/O in one thread with computations
in another. so i'm searching for fastest and easiest-to-implement
scheme. what you propose?


-- 
Best regards,
 Bulat                            mailto:bulatz at HotPOP.com