[Haskell-cafe] uvector package appendU: memory leak?

[Claus Reinke]

>>     IntMap (UArr (Word16 :*: Word8))
>>
>> I was adding elements to the map using something like:
>>
>>     v = map singletonU (a :*: b)
>>
>>     insertWith appendU k v m
>>
>> However doing this eats a *lot* of memory.

Since 'insertWith' doesn't actually do the 'appendU', the appends
will also be compressed in time, at point of use, rather than spread
out in time, over construction. Which might be inconvenient if large
volumes of data are involved.

>> So the question is: why appending an array of only one element to an  
>> existing array causes memory problems?
> 
> It must copy the entire array.

And doing this repeatedly, with arrays of increasing length, would
not be a good idea. For single-threaded use, one might use fusion
to avoid the construction/recopying of the intermediate arrays. 

As usual, if the single-threading happens in the body of a recursive
loop, compile-time fusion won't get a chance to work unless one
unfolds the recursion a few steps. But one could do similar fusion
dynamically, by partially reifying the construction functions (if we
are appending an array that is itself created via append, then a
single combined append will do). 

Wasn't there once a similar issue with strict bytestrings and cons?
Ah, found it - different mailing list:

http://www.haskell.org/pipermail/glasgow-haskell-users/2006-November/011603.html

Was this actually resolved? From a quick glance, it seems I
suggested runtime fusion, Don answered with compile-time
fusion, Duncan suggested a different hack to achieve runtime
fusion. But was the runtime fusion of nested cons in strict 
bytestring, or nested appends in uvector, ever implemented?
Does the stream-fusion framework handle that implicitly?

Claus