[Haskell-cafe] why does the binary library require so much memory?

[Don Stewart]

bos:
> On Fri, Jul 31, 2009 at 1:56 PM, Jeremy Shaw <jeremy at n-heptane.com> wrote:
> 
> 
>     Using encode/decode from Binary seems to permamently increase my
>     memory consumption by 60x fold. I am wonder if I am doing something
>     wrong, or if this is an issue with Binary.
> 
> 
> It's an issue with the Binary instance for lists, which forces the entire spine
> of the list too early. This gives you a gigantic structure to hold onto.

This is the current instance

    instance Binary a => Binary [a] where
        put l  = put (length l) >> mapM_ put l
        get    = do n <- get :: Get Int
                    getMany n

    -- | 'getMany n' get 'n' elements in order, without blowing the stack.
    getMany :: Binary a => Int -> Get [a]
    getMany n = go [] n
     where
        go xs 0 = return $! reverse xs
        go xs i = do x <- get
                     -- we must seq x to avoid stack overflows due to laziness in
                     -- (>>=)
                     x `seq` go (x:xs) (i-1)

It used to be this, though,

        xs <- replicateM n get     -- now the elems.


-- Don