new i/o library

[Bulat Ziganshin]

Hello Einar,

Friday, January 27, 2006, 4:19:55 PM, you wrote:

EK> One simple optimization is that you can omit all buffering with
EK> unbuffered operation. Then simply add the buffer (which is ok
EK> because Handles are mutable) if the user ever calls hLookAhead.

yes, i do it

>> moreover, i have an idea how to implement async i/o without complex
>> burecreacy: use mmapped files, may be together with miltiple buffers.
>> for example, we can allocate four 16kb buffers. when one buffer is
>> filled with written data, the program unmaps it and switches to use
>> the next buffer. i don't tested it, but OS can guess that unmapped
>> buffer now should be asynchronously written to disk. the same for
>> reading - when we completely read one buffer, we can unmap it, switch
>> to the second buffer and map the third so that the OS can
>> asynchronously fill the third buffer while we are reading second.
>> should this work, at least on the main desktop OSes?

EK> Please no. There are multiple reasons to avoid mmapped files.
EK> 1) They make very few performance guarantees for reading
EK>    (i.e. a Haskell thread touches memory which has not yet
EK>     been read from the file causing IO and all the other
EK>     Haskell threads are blocked too)

yes, it seems that using mmapped file may slowdown such program

EK> 2) The time of writes is unpredictable making implementing a
EK>    hFlush harder? (not sure about this)

i can say only about Windows - here FlushViewOfFile() do it

EK> 3) Not all file descriptors will support it - i.e. we will
EK>    need the read/write path in any case.

i don't understand what you mean, can you please explain futher?

EK> 4) Mmap cannot be used for random access for arbitrary files
EK>    since they may be larger than the address space. This means
EK>    some kind of window needs to be implemented - and this is
EK>    easily done with read/write.

that's not true, at least for Windows - see MapViewOfFile()



-- 
Best regards,
 Bulat                            mailto:bulatz at HotPOP.com