[Haskell-cafe] uvector package appendU: memory leak?

[Don Stewart]

manlio_perillo:
> Hi.
>
> As with a previous post, I think I have found a possible memory problem  
> with the uvector package.
>
> I have this data structure (for, again, my Netflix Prize project):
>
>     IntMap (UArr (Word16 :*: Word8))
>
> I was adding elements to the map using something like:
>
>     v = map singletonU (a :*: b)
>
>     insertWith appendU k v m
>
> However doing this eats a *lot* of memory.
>
> Today I have rewritten my program to use `alter` and `snocU`:
>
>     append Nothing = Just $ singletonU (a :*: b)
>     append (Just u) = Just $ snocU u (a :*: b)
>
>     alter append k m
>
>
> This, finally, works.
>
> Unfortunately I'm still not able to load the entire Netflix Prize  
> training data set, grouping ratings by customers, because my PC has only  
> 2 GB of RAM.
> The required memory is about 2 GB, but when the system start to swap, I  
> have to kill the program.
>
>
> So the question is: why appending an array of only one element to an  
> existing array causes memory problems?


It must copy the entire array.

-- Don