[Haskell-cafe] Aeson memory use

[Vanessa McHale]

I'm not 100% sure it's applicable in your case, but perhaps trying
threadscope would be a good idea?

Based on the heap profile, I'm guessing that most of the allocations are
due to slicing ByteStrings up. What happens when you try to read a map
of Integers?

I'd also note that your example uses evaluate and performGC which isn't
entirely realistic and thus may not actually correspond to the
performance of the application in practice. I gather it *was* a problem
in practice but nonetheless it's always good to benchmark the right thing.

On 08/08/2018 02:54 PM, Lana Black wrote:
> Hello Cafe,
>
> Recently while trying to optimize some code, I noticed that aeson
> requires quite a substantial amount of memory when parsing map-like
> structures. The quick example I crafted [1] is quite surprising. In
> order to parse an 11 megabyte file aeson requires around 300 megabytes
> of memory (according to the heap profiler), and the total memory
> footprint (according to htop) jumps to around 700 megabytes.
>
> Is there a way to reduce this number to something less insane? I
> suspect that I might be doing something wrong here, but I can't
> exactly see what.
>
> [1] https://github.com/greydot/aeson-example
> _______________________________________________
> Haskell-Cafe mailing list
> To (un)subscribe, modify options or view archives go to:
> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
> Only members subscribed via the mailman list are allowed to post.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: OpenPGP digital signature
URL: <http://mail.haskell.org/pipermail/haskell-cafe/attachments/20180808/d3d44c8c/attachment.sig>