[Haskell-cafe] Re: Mining Twitter data in Haskell and Clojure

[Ketil Malde]

braver <deliverable at gmail.com> writes:

> In fact, the tag cafe2, when run on the full dataset, gets stuck at 11
> days, with RAM slowly getting into 50 GB

One tip might be to limit available heap memory by using +RTS -M2G (or
whatever your real memory is).  If (as seems likely) the RAM usage leads to
thrashing (the symptoms being 'top' showing substantially less than 100%
CPU usage, and a less responsive system), limiting heap will cause your
program to fail faster, which is always an advantage when debugging.

Unless you actually expect the working set to be fifty gig, that is.

-k
-- 
If I haven't seen further, it is by standing in the footprints of giants