[Haskell-cafe] ST not strict enough?

[Jason Dusek]

2011/11/15 Roman Cheplyaka <roma at ro-che.info>:
> * Jason Dusek <jason.dusek at gmail.com> [2011-11-15 20:08:48+0000]
>> I'm having some trouble with memory usage in rebuilding a
>> ByteString with some sequences escaped. I thought I'd try
>> vectors. However, it seems that even a relatively simple
>> function, one that places all the bytes of a ByteString in to a
>> vector, uses a great deal of memory.
>
> I think what's happening here is ByteString's "strictness" makes things
> actually lazy on your side.
>
> Namely, unpack function produces its result "strictly", whole list at
> once. As a result, the resulting list cannot be consumed one-by-one,
> so it takes memory. You see ST thunks because
>
>  mapM_ f as =  sequence_ (map f as)
>
> and that map probably gets fused with unpack.
>
> I guess the proper solution here is to use lazy bytestring and make sure
> the chunks are not very big.

Hi Roman,

Switching to the lazy ByteStrings API does, indeed, help; total
memory usage is around 16M. I will have a look at the rules that
are fired to see what I can learn.

--
Jason Dusek
()  ascii ribbon campaign - against html e-mail
/\  www.asciiribbon.org   - against proprietary attachments