[Haskell-cafe] Data.Binary stack overflow with Data.Sequence String
gwern0 at gmail.com
Tue Mar 3 23:30:24 EST 2009
So recently I've been having issues with Data.Binary & Data.Sequence;
I serialize a 'Seq String'
You can see the file here: http://code.haskell.org/yi/Yi/IReader.hs
The relevant function seems to be:
-- | Read in database from 'dbLocation' and then parse it into an 'ArticleDB'.
readDB :: YiM ArticleDB
readDB = io $ (dbLocation >>= r) `catch` (\_ -> return empty)
where r x = fmap (decode . BL.fromChunks . return) $ B.readFile x
-- We read in with strict bytestrings to guarantee the
file is closed,
-- and then we convert it to the lazy bytestring
-- This is inefficient, but alas...
My current serialized file is about 9.4M. I originally thought that
the issue might be the recent upgrade in Yi to binary 0.5, but I
unpulled patches back to past that, and the problem still manifested.
Whenever yi tries to read the articles.db file, it stack overflows. It
actually stack-overflowed on even smaller files, but I managed to bump
the size upwards, it seems, by the strict-Bytestring trick.
Unfortunately, my personal file has since passed whatever that limit
I've read carefully the previous threads on Data.Binary and Data.Map
stack-overflows, but none of them seem to help; hacking some $!s or
seqs into readDB seems to make no difference, and Seq is supposed to
be a strict datastructure already! Doing things in GHCi has been
tedious, and hasn't enlightened me much: sometimes things overflow and
sometimes they don't. It's all very frustrating and I'm seriously
considering going back to using the original read/show code unless
anyone knows how to fix this - that approach may be many times slower,
but I know it will work.
More information about the Haskell-Cafe