[Haskell-cafe] ST not strict enough?

[Jason Dusek]

Hi All,

I'm having some trouble with memory usage in rebuilding a
ByteString with some sequences escaped. I thought I'd try
vectors. However, it seems that even a relatively simple
function, one that places all the bytes of a ByteString in to a
vector, uses a great deal of memory.

I've pulled this function out into a mini-module for testing:

  https://github.com/solidsnack/arx/blob/ST/Rebuild.hs#L37

On a one megabyte input, it peaks at ~38M of memory:

  :;  dd if=/dev/zero bs=1M count=1 | ./rebuildprof +RTS -s
  ...
        38,724,208 bytes maximum residency (5 sample(s))
         1,983,720 bytes maximum slop
                76 MB total memory in use (0 MB lost due to fragmentation)
  ...
    %GC time      61.3%  (61.5% elapsed)

A heap profile by type, with -hy, shows a linear rise in ST
items -- up to ~26M -- and then a linear decrease. It would
stand to reason that, with sufficient strictness, the memory
allocated to ST would stay constant and small.

Should I be annotating my functions with strictness, for the
vector reference, for example? Should I be using STUArrays,
instead?

--
Jason Dusek
()  ascii ribbon campaign - against html e-mail
/\  www.asciiribbon.org   - against proprietary attachments