[Haskell-cafe] ST not strict enough?
roma at ro-che.info
Tue Nov 15 22:17:11 CET 2011
* Jason Dusek <jason.dusek at gmail.com> [2011-11-15 20:08:48+0000]
> I'm having some trouble with memory usage in rebuilding a
> ByteString with some sequences escaped. I thought I'd try
> vectors. However, it seems that even a relatively simple
> function, one that places all the bytes of a ByteString in to a
> vector, uses a great deal of memory.
> Should I be annotating my functions with strictness, for the
> vector reference, for example? Should I be using STUArrays,
I think what's happening here is ByteString's "strictness" makes things
actually lazy on your side.
Namely, unpack function produces its result "strictly", whole list at
once. As a result, the resulting list cannot be consumed one-by-one,
so it takes memory. You see ST thunks because
mapM_ f as = sequence_ (map f as)
and that map probably gets fused with unpack.
I guess the proper solution here is to use lazy bytestring and make sure
the chunks are not very big.
Roman I. Cheplyaka :: http://ro-che.info/
More information about the Haskell-Cafe