[Haskell-cafe] two problems with Data.Binary and Data.ByteString

[Bryan O'Sullivan]

On Tue, Aug 12, 2008 at 5:13 PM, Tim Newsham <newsham at lava.net> wrote:

> The data type I'm storing is a Map (of maps):
>
>   type DailyDb = M.Map Date Daily
>   type InstrsDb = M.Map String DailyDb
>
> What's going on here?

The default marshalling scheme that Binary uses for lists and maps
(which are flattened to lists before writing out) is not streamable.
Instead of writing out data in chunks, it computes the length of the
list and writes that, followed by the elements. Presumably on the read
side, a huge thunk is being built up before any actual Map creation
starts.

> I also noticed another issue while testing.  If my program loads
> the data at startup by calling loadState then all later calls to
> saveState give an error:
>
>  Log: savedState.bin: openFile: resource busy (file is locked)
>
> this does not occur if the program wasnt loaded.

Your loading of state isn't being forced to complete, so the file
handle is still open when you try to save to the same file. The H98
standard requires that file handles be locked for exclusive access
during writes.

To force the read to finish, use rnf. You can find a description of
how to use it, and the typeclasses involved, here:
http://book.realworldhaskell.org/beta/concurrent.html