[Haskell-cafe] possible memory leak in uvector 0.1.0.3

[Claus Reinke]

>>> I split the string in lines, then map some functions on each line to 
>>> parse the data, and finally calling toU, for converting to an UArr.
>> 
>> Just to make sure (code fragments or, better, reduced examples
>> would make it easier to see what the discussion is about): are you 
>> forcing the UArr to be constructed before putting it into the Map?
> 
> parse handle =
>   contents <- S.hGetContents handle
>   let v =  map singleton' $ ratings contents
>   let m = foldl1' (unionWith appendU) v
>   v `seq` return $! m
> 
>   where
>     -- Build a Map with a single movie rating
>     singleton' :: (Word32, Word8) -> MovieRatings
>     singleton' (id, rate) =
>       singleton (fromIntegral $ id) (singletonU $ pairS (id, rate))

That helps to make things clearer, I think. One issue is
the nature of Maps (strict in keys, non-strict in values).

- neither singleton nor unionWith are strict in the Map values, so
    nothing here forces the evaluation of rate or construction 
    of UArr

    Prelude Data.IntMap> (unionWith (++) (singleton 1 undefined) (singleton 2 undefined)) `seq` ()
    ()

- singletonU is strict, but that only means that it will evaluate its
    parameter if it is evaluated itself (which it isn't, because 
    singleton isn't strict)

- seq on a list only forces the first node of the list ((:),[],_|_),
    so (v `seq`) isn't likely to help much. Also, you probably
    do not want to force the whole list of singletons before
    builing the Map, you want the singletons to be constructed 
    and consumed incrementally.

- forcing a Map doesn't force any of the values, nor does
    it force more than the top-level node of whatever the
    internal Map representation is, so (return $! m) isn't much
    help, either (by nature of unionWith and foldl1', it'll force 
    all keys before it can say anything much about the Map,
    but the values remain untouched, burried further under
    unevaluated (++)s)
 
> type Rating = Word32 :*: Word8
> type MovieRatings = IntMap (UArr Rating) -- UArr from uvector

A standard trick to keep Map values evaluated by construction
is to make the availability of keys dependent on their values, eg
(singleton key) $! value. That won't help with unionWith and
the appendUs, but it should allow the source string references
to be dropped early, as the singletons are constructed.

Hth,
Claus