[Haskell-cafe] uvector package appendU: memory leak?

Manlio Perillo manlio_perillo at libero.it
Sun Mar 29 15:17:24 EDT 2009


Hi.

As with a previous post, I think I have found a possible memory problem 
with the uvector package.

I have this data structure (for, again, my Netflix Prize project):

     IntMap (UArr (Word16 :*: Word8))

I was adding elements to the map using something like:

     v = map singletonU (a :*: b)

     insertWith appendU k v m

However doing this eats a *lot* of memory.

Today I have rewritten my program to use `alter` and `snocU`:

     append Nothing = Just $ singletonU (a :*: b)
     append (Just u) = Just $ snocU u (a :*: b)

     alter append k m


This, finally, works.

Unfortunately I'm still not able to load the entire Netflix Prize 
training data set, grouping ratings by customers, because my PC has only 
2 GB of RAM.
The required memory is about 2 GB, but when the system start to swap, I 
have to kill the program.


So the question is: why appending an array of only one element to an 
existing array causes memory problems?

This should be pratically the same as adding an element.



Thanks  Manlio


More information about the Haskell-Cafe mailing list