getting a grip on memory usage
Wed, 22 May 2002 13:21:20 -0500
On Wednesday 22 May 2002 12:48 pm, Hal Daume III wrote:
> so i have a function that is eating *tons* of memory. my application
> is clustering and i put everything in UArrays or Arrays. for
> clustering 5 data points with 4 features (a tiny tiny set) the
> program gobbles 350mbs! most of the usage is coming from this
> distance function:
Ahh, a fellow memory abuser. ;)
I recently ran into very similar behavior while implementing a Haskell=20
rendering engine. I was using a lookup table deep within a loop, and=20
references to the lookup table seemed to be allocated as each=20
computation was deferred. I immediately allocated over 150MB for a=20
very simple render.
Try turning on heap profiling, and see if you get anything interesting. =20
There is an option to display "named" references (or something like=20
that) that might provide insight into what's responsible.
In my case, I rewrote my monad to be stricter (and used case), and=20
forced a result data structure to be strict in its components. This=20
forced complete evaluation of all of the elements prior to returning=20
the result. I don't know if you have a nice place to put something=20
like that, but it's worth a look.
When all was said and done, my application memory usage never went above=20
4MB, which is exactly where I expected it to be.
Check out this link:
It has some interesting ways of forcing evaluation at various times,=20
etc., although I prefer other ways of obtaining these results. =20
However, it is a good list of candidate changes for debugging.
The river is moving.
The blackbird must be flying.