[ANN] Memory mapped files for POSIX and Windows

David Roundy droundy at darcs.net
Wed Mar 19 11:34:21 EDT 2008

On Tue, Mar 18, 2008 at 11:03:23PM +0300, Bulat Ziganshin wrote:
> Hello David,
> Tuesday, March 18, 2008, 10:15:48 PM, you wrote:
> > Incidentally, you'll probably find that for large files, using mmap on
> > Windows is a huge loser when compared with lazy bytestrings.  As far as I
> > understand, on Windows when a file is mmapped, its entire contents are
> > immediately loaded into memory, so if you mmap a file that is larger than
> > your memory, you pay a huge penalty on Windows.
> it's definitely not true, at least for my own experiments. although i
> never tried o map entire file to memory, but used only small window
> into file. mmaping for windows and linux implemented in
> http://www.haskell.org/library/StreamsBeta.tar.gz where it used just
> to make i/o faster. at least i tried but found undesirable effects,
> but not one you have mentioned

I've never tried this (since I don't have windows), but it's what I've been
told.  And no, you wouldn't have noticed unless you tried to mmap a large
file in its entirety.  It's a major difference in behavior, though, between
posix and windows mmap.  The former behaves essentially like a faster,
better version of lazy IO (provided you don't try to modify the file
in-place) that is a major improvement when under tight memory pressure
(because it's file-backed memory).  The latter is quite the opposite.
David Roundy
Department of Physics
Oregon State University

More information about the Libraries mailing list