[Haskell-cafe] uvector package appendU: memory leak?
Manlio Perillo
manlio_perillo at libero.it
Sun Mar 29 15:17:24 EDT 2009
Hi.
As with a previous post, I think I have found a possible memory problem
with the uvector package.
I have this data structure (for, again, my Netflix Prize project):
IntMap (UArr (Word16 :*: Word8))
I was adding elements to the map using something like:
v = map singletonU (a :*: b)
insertWith appendU k v m
However doing this eats a *lot* of memory.
Today I have rewritten my program to use `alter` and `snocU`:
append Nothing = Just $ singletonU (a :*: b)
append (Just u) = Just $ snocU u (a :*: b)
alter append k m
This, finally, works.
Unfortunately I'm still not able to load the entire Netflix Prize
training data set, grouping ratings by customers, because my PC has only
2 GB of RAM.
The required memory is about 2 GB, but when the system start to swap, I
have to kill the program.
So the question is: why appending an array of only one element to an
existing array causes memory problems?
This should be pratically the same as adding an element.
Thanks Manlio
More information about the Haskell-Cafe
mailing list