[Haskell-cafe] Re: Constructing Data.Map from ByteString

[Dave Tapley]

Just a few updates on this one:

I've upgraded to bytestring-0.9.0.5 from Darcs, no improvement.
Also this morning I tried using Data.HashMap with Bytestring's readInt
and HashMap's hashInt.. The result was a Stack space overflow :(

God damn it I don't want to have to go back to C++ but soon will have
no choice with time constraints :(

Dave,


On 10/03/2008, Dave Tapley <dave.a.tapley at gmail.com> wrote:
> Hi all,
>
>  I've been plugging away at this all day and some discussion in
>  #haskell has been fruitless. Perhaps you have the inspiration to see
>  what's happening!
>
>  Concerning this minimal example:
>  http://hpaste.org/6268
>
>  It works as required, loading K/V pairs into a Data.Map, the concern
>  is the amount of memory used. Pausing (using a getLine) after the
>  readFile one can see (through 'ps v') that the whole file 'f' is
>  loaded in to memory.
>  However once the Map is created the memory footprint swells to around
>  five times the size. Surely this can't be just overhead from Map?
>  My reasoning was that by using ByteString and getting the whole file
>  into memory a small and linear increase would be seen for Map
>  overhead..
>
>  I have tried using both Data.ByteString.Char8 and
>  Data.ByteString.Lazy.Char8 with negligible difference.
>  For a hoot I tried it with String and yes, it's ridiculous :)
>
>  Cheers,
>
> Dave
>