[Haskell-cafe] help optimizing memory usage for a program
bulat.ziganshin at gmail.com
Mon Mar 2 11:14:35 EST 2009
Monday, March 2, 2009, 6:30:51 PM, you wrote:
> The process-data-1 program parse the entire dataset using about 1.4 GB
> of memory (3x increment).
> This is strange.
> The memory required is proportional to the number of ratings.
> It may be IntMap the culprit, or the garbage collector than does not
> release the memory to the operating system (or, worse, does not
> deallocate all used temporary memory).
ghc has 2 garbage collectors. first is copying, usied by default
(because it's faster): when previously allocated memory block
overflows, it collects used data by copying them into *new* data block
allocated from OS and then use old data block to satisfy further
let's calculate. if at GC moment your program has allocated 100 mb of
memory and only 50 mb was not a garbage, then memory usage will be 150
another GC is compacting one - it's about 2x slower, but collects data
in-place. moreover, you may set up"growing factor". with a g.f. of
1.5, for example, memory will be collected once heap will become 1.5x
larger than real memory usage after last GC. this effectively
guarantees that memory overhead will never be over this factor
look at GHC manual, RTS switches section. and last - GHC never returns
memory to the OS, there should be ticket on this
Bulat mailto:Bulat.Ziganshin at gmail.com
More information about the Haskell-Cafe