[Haskell-cafe] Too much strictness in binary-0.5.0.2

[Alexey Khudyakov]

> Yes. We decided that having the Get monad as a lazy state monad just
> doesn't make sense. It makes this one use case work, but causes more
> problems generally. Certainly we need to be able to do lazy binary
> deserialisation too, but our feeling is that it should be more explicit
> and integrate better with error handling. Using a lazy state monad gives
> us neither.

This is valid point. I think I'll just use variant with explicit laziness.


> Note also that if we changed runGetState to do any error handling that
> this would also break the lazy state monad properties that you were
> previously relying upon.

I'd appreciate addition of error handling very much. Currently I have
to use ErrorT and do checks by hands. I have very simple parsers so
it's possible. It's not possible to do such things in general.

Trading laziness for error handling is good thing I believe. At the
moment it's hard to decode possibly malformed data.


On 10/17/09, Duncan Coutts <duncan.coutts at googlemail.com> wrote:
> On Sat, 2009-09-19 at 00:36 +0400, Khudyakov Alexey wrote:
>> Hello
>>
>> I run into problems with new binary package. Following function reads a
>> list
>> of elements one by one until end of stream. List is very long (won't fit
>> into
>> memory).
>
> Sorry for the late reply.
>
>> In binary-0.5.0.1 and earlier it read list lazily. Now it seems that it
>> tries
>> to read whole list to memory. Program does not produce any output and
>> memory
>> usage steadily grows.
>
> Yes. We decided that having the Get monad as a lazy state monad just
> doesn't make sense. It makes this one use case work, but causes more
> problems generally. Certainly we need to be able to do lazy binary
> deserialisation too, but our feeling is that it should be more explicit
> and integrate better with error handling. Using a lazy state monad gives
> us neither.
>
>> > getStream :: Get a -> Get [a]
>> > getStream getter = do
>> >   empty <- isEmpty
>> >   if empty
>> >     then return []
>> >     else do x <- getter
>> >             xs <- getStream getter
>> >             return (x:xs)
>>
>> How could I add laziness to this function to revert to old behavior.
>
> You can make it explicitly lazy using something like:
>
> getStream :: Get a -> BS.ByteString -> [a]
> getStream getter bs = unfoldr step (bs, 0)
>   where
>     step (bs, off) = case runGetState getOne bs off of
>       (Nothing, _,   _   ) -> Nothing
>       (Just x,  bs', off') -> Just (x, (bs', off'))
>
>     getOne = do
>       empty <- isEmpty
>       if empty
>         then return Nothing
>         else fmap Just getter
>
> Note the distinct lack of error handling, but if we had runGetState
> return an Either then you could still make this code lazy, but you'd
> have to decide what to do when you got a decoding error. You could at
> this point translate it into a pure error, or enrich your output stream
> type to account for the possibility of errors.
>
> Note also that if we changed runGetState to do any error handling that
> this would also break the lazy state monad properties that you were
> previously relying upon.
>
> Duncan
>
>