[Haskell-cafe] two problems with Data.Binary and Data.ByteString

[Ketil Malde]

Old threads never die:

Tim Newsham <newsham at lava.net> writes:

>> Chunk = {
>>    length :: Word8
>>    elems :: [Elem]  --  0..255 repetitions
>>  }
>> Chunks = [Chunk] -- terminated with the first 0 length Chunk

> I tried my hand at the encoding above:
>
>     http://www.thenewsh.com/%7Enewsham/store/test10.hs
>
> it seems to work, although it doesn't seem to be very efficient.
> I'm getting very large memory growth when I was hoping it
> would be lazy and memory efficient...  What's leaking?

Did you ever get to the bottom of this?

I have a similar problem with Data.Binary that I don't know how to
work around yet.  It boils down to reading a large list.  This
exhibits the problem:

  newtype Foo = Foo [Word8]
  instance Binary Foo where
      get = do 
        xs <- replicateM 10000000 get
        return (Foo xs)

Doing 'x <- decodeFile "/dev/zero" and "case x of Foo y -> take 10 y"
blows the heap.  I thought Data.Binary was lazy?

My actual program looks something like this:

  instance Binary MyData where
         get = do
             header <- get
             data   <- replicateM (data_length header) $ do 
                                  ....stuff to read a data item
             return (MyData header data)

This blows the stack as soon as I try to access anything, even if it's
just the contents of the header.  Why?

My understanding of how Data.Binary works must be sorely lacking.
Could some kind soul please disperse some enlightenment in my
direction? 

-k
-- 
If I haven't seen further, it is by standing in the footprints of giants