[Haskell-cafe] possible memory leak in uvector 0.1.0.3

[Duncan Coutts]

On Tue, 2009-03-03 at 02:12 +0100, Manlio Perillo wrote:
> Hi.
> 
> In the "help optimizing memory usage for a program" I discovered some 
> interesting things:
> 
> 
> 1) Using lazy ByteStrings to read files it not a good choice, since the
>     garbage collector is not able to proper garbage cleanup.
> 
>     Running with -F1 RTS flag, however, keeps memory usage down.

It is certainly possible to have proper garbage cleanup. I can write
programs using lazy ByteStrings that process many megabytes of data and
yet run in 1Mb of heap space.

At first guess it sounds like you're holding onto too much, if not the
whole stream perhaps bits within each chunk. Each chunk read from the
file is 64k big and keeping any substring will force the whole chunk to
be retained. If you're only keeping a fraction of each chunk you can use
the ByteString.Lazy.copy function to make a deep copy and let the
original 64k chunk get collected.

Duncan