[Haskell-cafe] Aeson memory use
lanablack at amok.cc
Thu Aug 9 12:57:15 UTC 2018
On 08/08/2018 08:24 PM, Vanessa McHale wrote:
> I'm not 100% sure it's applicable in your case, but perhaps trying
> threadscope would be a good idea?
> Based on the heap profile, I'm guessing that most of the allocations are
> due to slicing ByteStrings up. What happens when you try to read a map
> of Integers?
Parsing a 'Map Int Int' gives a similar picture, with slightly lower
memory use due to the absence of [Text]. However, it is still in the
hundreds of megabytes for a 10 megabyte file. I also noticed that Map
itself contributes significantly to the memory footprint.
> I'd also note that your example uses evaluate and performGC which isn't
> entirely realistic and thus may not actually correspond to the
> performance of the application in practice. I gather it *was* a problem
> in practice but nonetheless it's always good to benchmark the right thing.
Removing evaluate and force adds THUNKs to the heap profile, which is
probably the wrong thing to benchmark anyway. What I was trying to
benchmark was the memory footprint during the parsing stage and after
it, when only the resulting data should be residing in memory.
More information about the Haskell-Cafe