[Haskell-cafe] uvector package appendU: memory leak?
manlio_perillo at libero.it
Sun Mar 29 15:17:24 EDT 2009
As with a previous post, I think I have found a possible memory problem
with the uvector package.
I have this data structure (for, again, my Netflix Prize project):
IntMap (UArr (Word16 :*: Word8))
I was adding elements to the map using something like:
v = map singletonU (a :*: b)
insertWith appendU k v m
However doing this eats a *lot* of memory.
Today I have rewritten my program to use `alter` and `snocU`:
append Nothing = Just $ singletonU (a :*: b)
append (Just u) = Just $ snocU u (a :*: b)
alter append k m
This, finally, works.
Unfortunately I'm still not able to load the entire Netflix Prize
training data set, grouping ratings by customers, because my PC has only
2 GB of RAM.
The required memory is about 2 GB, but when the system start to swap, I
have to kill the program.
So the question is: why appending an array of only one element to an
existing array causes memory problems?
This should be pratically the same as adding an element.
More information about the Haskell-Cafe