[Haskell-cafe] Aeson memory use

Lana Black lanablack at amok.cc
Thu Aug 9 12:57:15 UTC 2018

On 08/08/2018 08:24 PM, Vanessa McHale wrote:
> I'm not 100% sure it's applicable in your case, but perhaps trying
> threadscope would be a good idea?
> Based on the heap profile, I'm guessing that most of the allocations are
> due to slicing ByteStrings up. What happens when you try to read a map
> of Integers?

Parsing a 'Map Int Int' gives a similar picture, with slightly lower 
memory use due to the absence of [Text]. However, it is still in the 
hundreds of megabytes for a 10 megabyte file. I also noticed that Map 
itself contributes significantly to the memory footprint.

> I'd also note that your example uses evaluate and performGC which isn't
> entirely realistic and thus may not actually correspond to the
> performance of the application in practice. I gather it *was* a problem
> in practice but nonetheless it's always good to benchmark the right thing.

Removing evaluate and force adds THUNKs to the heap profile, which is 
probably the wrong thing to benchmark anyway. What I was trying to 
benchmark was the memory footprint during the parsing stage and after 
it, when only the resulting data should be residing in memory.

More information about the Haskell-Cafe mailing list