[Haskell-cafe] ST not strict enough?

Roman Cheplyaka roma at ro-che.info
Tue Nov 15 22:17:11 CET 2011

* Jason Dusek <jason.dusek at gmail.com> [2011-11-15 20:08:48+0000]
> I'm having some trouble with memory usage in rebuilding a
> ByteString with some sequences escaped. I thought I'd try
> vectors. However, it seems that even a relatively simple
> function, one that places all the bytes of a ByteString in to a
> vector, uses a great deal of memory.
> Should I be annotating my functions with strictness, for the
> vector reference, for example? Should I be using STUArrays,
> instead?

Hi Jason,

I think what's happening here is ByteString's "strictness" makes things
actually lazy on your side.

Namely, unpack function produces its result "strictly", whole list at
once. As a result, the resulting list cannot be consumed one-by-one,
so it takes memory. You see ST thunks because

  mapM_ f as =  sequence_ (map f as)

and that map probably gets fused with unpack.

I guess the proper solution here is to use lazy bytestring and make sure
the chunks are not very big.

Roman I. Cheplyaka :: http://ro-che.info/

More information about the Haskell-Cafe mailing list