Streams 0.2.1 beta released

[Bulat Ziganshin]

Hello David,

Wednesday, July 12, 2006, 4:51:26 PM, you wrote:
>> Using memory-mapped files provides slight speed improvement because it
>> excludes the need to move data between OS buffer and application buffer.
>> Moreover, this may scale better in multi-threading environments without
>> need to implement special async I/O, although i don't checked this
>> assumption.

> There's also the potentially quite significant advantage that with
> mmapped files you can open up files much larger than the physical
> memory size.

you can do the same with regular files

> Or if you open smaller files, but the system begins to
> swap, mmaped files don't have to be swapped to a swap file, which
> makes things more efficient in general.  In my opinion this is a much
> more important reason to use mmap than avoiding a memory copy from
> kernel space to user space.

you mean reading from thousands of small files files simultaneously?

on windows using mmaped files has the disadvantage of swapping out in
first place everything other! so when you write to such file, say, 1gb
of data and have 1 gb or less of memory, all the programs will swapped
to disk!

anyway, now Streams library contains simple but powerful and
OS-independent interface to mmaped files that can be used beyond my
own support for mmaped file as Stream instance. The whole interface is:


-- | Memory-mapped file handle. It's just FD on Unix
type MemoryMappedFD

-- | Open file handle 'FD' opened in read/write mode 'IOMode' for following mapping operations
-- and return mapping handle 'MemoryMappedFD'. In Unix it just returns FD itself
myOpenMMap :: FD -> IOMode -> IO MemoryMappedFD

-- | Map 'Int' bytes of file starting from position 'Integer' and return address of mapped buffer.
-- Mapping done for file with handle 'MemoryMappedFD' open in read/write mode 'IOMode'.
myMMap :: MemoryMappedFD -> IOMode -> Integer -> Int -> IO (Ptr Word8)

-- | Flush mmapped buffer with address 'Ptr' and size 'Int'
myFlushMMap  :: Ptr Word8 -> Int -> IO ()

-- | Unmap buffer with address 'Ptr' and size 'Int'
myUnMMap :: Ptr Word8 -> Int -> IO ()

-- | Close "mapping handle". In Unix it does nothing
myCloseMMap :: MemoryMappedFD -> IO ()


(although the function names looks a bit stupid ;) )



-- 
Best regards,
 Bulat                            mailto:Bulat.Ziganshin at gmail.com