[Haskell-cafe] OS swapping and haskell data structures
alex at alexjacobson.com
Wed Aug 1 15:32:54 EDT 2007
Ok, so for low throughput applications, you actually need a disk
strategy. Got it.
Ok, is there a standard interface to BerkleyDB or some other disk based
Duncan Coutts wrote:
> On Wed, 2007-08-01 at 11:31 -0700, Bryan O'Sullivan wrote:
>> Alex Jacobson wrote:
>>> If you create a Data.Map or Data.Set larger than fits in physical
>>> memory, will OS level swapping enable your app to behave reasonably or
>>> will things just die catastrophically as you hit a memory limit?
>> Relying on the OS to page portions of your app in and out should always
>> be the fallback of last resort. You are fairly guaranteed to get
>> terrible performance because the VM subsystem can't anticipate your
>> app's memory access patterns, and catastrophic death of either your app
>> or other system processes is a strong possibility (Google for "OOM
>> killer" if you want some horror stories). In many cases, you can't even
>> rely on paging being possible.
> Furthermore, as I understand it, GC does not interact well with paging
> since the GC has to traverse the data structures on major GCs it'll
> force it all to be kept in memory.
More information about the Haskell-Cafe