[Haskell-cafe] iteratee-compress space leak?

Michael A Baikov pacak at bk.ru
Fri Feb 18 15:27:19 CET 2011


I am trying to play with iteratee making parser for squid log files, but found that my code do not run in constant space when it tries to process compressed log files. So i simplified my code down to this snippet:

import Data.ByteString (ByteString)
import Data.Iteratee as I
import Data.Iteratee.Char
import Data.Iteratee.ZLib
import System

main = do
        args <- getArgs
        let fname     = args !! 0
        let blockSize = read $ args !! 1

        fileDriver (leak blockSize) fname >>= print

leak :: Int -> Iteratee ByteString IO ()
leak blockSize = joinIM $ enumInflate GZip defaultDecompressParams chunkedRead
    where
        consChunk :: Iteratee ByteString IO String
        consChunk = (joinI $ I.take blockSize I.length) >>= return . show

        chunkedRead :: Iteratee ByteString IO ()
        chunkedRead = joinI $ convStream consChunk printLines


First argument - file name (/var/log/messages.1.gz will do)
second - size of block to consume input. with low size (10 bytes) of consumed blocks it leaks very fast, with larger blocks (~10000) it works almost without leaks.

So. Is it bugs within my code, or iteratee-compress should behave differently?



More information about the Haskell-Cafe mailing list