[Haskell-cafe] Lazy object deserialization

[Scott Lawrence]

I tried it, but it still goes and reads the whole list. Looking at the 
`binary` package source code it seems that strict evaluation is hard-coded in 
a few places, presumably for performance reasons. It also seems to necessarily 
read the bytestring sequentially, so complex tree-like data structures would 
presumably encounter problems even if it worked for a list.

Ah well. As long as I'm not duplicating someone else's work, I'm more than 
happy to go at this from scratch.

On Wed, 13 Mar 2013, Jeff Shaw wrote:

> On 3/13/2013 12:15 AM, Scott Lawrence wrote:
>> Hey all,
>> 
>> All the object serialization/deserialization libraries I could find (pretty 
>> much just binary and cereal) seem to be strict with respect to the actual 
>> data being serialized. In particular, if I've serialized a large [Int] to a 
>> file, and I want to get the first element, it seems I have no choice but to 
>> deserialize the entire data structure. This is obviously an issue for large 
>> data sets.
>> 
>> There are obvious workarounds (explicitly fetch elements from the 
>> "database" instead of relying on unsafeInterleaveIO to deal with it all 
>> magically), but it seems like it should be possible to build a cereal-like 
>> library that allows proper lazy deserialization. Does it exist, and I've 
>> just missed it?
>> 
>> Thanks,
>> 
> I haven't tested this, but I suspect something like this could give you lazy 
> binary serialization and deserialization. It's not tail recursive, though.
>
> newtype LazyBinaryList a = LazyBinaryList [a]
>
> instance Binary a => LazyBinaryList a where
>    put (LazyBinaryList []) = putWord8 0
>    put (LazyBinaryList (x:xs)) = putWord8 1 >> put x >> put (LazyBinaryList 
> xs)
>    get = do
>        t <- getWord8
>        case t of
>            0 -> return (LazyBinaryList [])
>            1 -> do
>                x <- get
>                (LazyBinaryList xs) <- get
>                return $ LazyBinaryList (x:xs)
>

-- 
Scott Lawrence