[Haskell-cafe] High memory usage with 1.4 Million records?
nominolo at googlemail.com
Fri Jun 8 21:17:51 CEST 2012
On 8 June 2012 01:39, Andrew Myers <asm198 at gmail.com> wrote:
> Hi Cafe,
> I'm working on inspecting some data that I'm trying to represent as records
> in Haskell and seeing about twice the memory footprint than I was
That is to be expected in a garbage-collected language. If your
program requires X bytes of memory then allocators will usually
trigger garbage collection once the heap reaches a size of 2X bytes.
If it didn't do this then every allocation would require a GC. You
can change this factor with +RTS -F option. E.g., +RTS -F1.5 should
reduce this to only 50% overhead, but will trigger more frequent
garbace collections. To find the actual residency (live data) see the
output of +RTS -s
There may still be room for improvement. For example, you could try
turning on the compacting GC -- which trades GC performance for lower
memory usage. You can enable it with +RTS -c
The reason that -hc runs slowly is that it performs a GC every 1s (I
think). You can change this using the -i option. E.g. -i60 only
examines the heap every 60s. It will touch almost all your live data,
so it is an inherently RAM-speed bound operation.
More information about the Haskell-Cafe