[Haskell-cafe] possible memory leak in uvector 0.1.0.3

[Kenneth Hoste]

On Mar 3, 2009, at 11:10 , Manlio Perillo wrote:

> Manlio Perillo ha scritto:
>> [...]
>>>> The other program, with a lot of array concatenations, still eats  
>>>> a lot  of memory...
>>>
>>> Concatenating arrays generally copies data. Which uses memory.
>>>
>> Of course, but why the garbage collector does not "release" this  
>> temporary used memory?
>> Note that I'm using the -F1 flag with the RST.
>> Maybe it is a problem with IntMap, when there are a lot of keys?
>
> It *is* a problem with IntMap.
> I have changed the program to not use any array concatenation, and  
> it still requires a lot of memory.
>
>
> Does esist a data structure that is able to store something like  
> 480189 keys with efficient memory usage?

I ran into the same problem when I first organized the IntMap to use  
user IDs as keys...

The problem is the huge amount of keys, and the small UArrays as values.

The overhead of both the IntMap and the UArray data types is just way  
too big with 480k different keys...

I never looked into it thoroughly, but if you look at the definition  
of IntMap, each key causes several words
of overhead, along with one word or so for each UArray.

K.

-- 

Kenneth Hoste
Paris research group - ELIS - Ghent University, Belgium
email: kenneth.hoste at elis.ugent.be
website: http://www.elis.ugent.be/~kehoste
blog: http://boegel.kejo.be