[Haskell-cafe] Aeson memory use

[Lana Black]

On 08/08/2018 08:24 PM, Vanessa McHale wrote:
> I'm not 100% sure it's applicable in your case, but perhaps trying
> threadscope would be a good idea?
> 
> Based on the heap profile, I'm guessing that most of the allocations are
> due to slicing ByteStrings up. What happens when you try to read a map
> of Integers?

Parsing a 'Map Int Int' gives a similar picture, with slightly lower 
memory use due to the absence of [Text]. However, it is still in the 
hundreds of megabytes for a 10 megabyte file. I also noticed that Map 
itself contributes significantly to the memory footprint.

> I'd also note that your example uses evaluate and performGC which isn't
> entirely realistic and thus may not actually correspond to the
> performance of the application in practice. I gather it *was* a problem
> in practice but nonetheless it's always good to benchmark the right thing.
> 

Removing evaluate and force adds THUNKs to the heap profile, which is 
probably the wrong thing to benchmark anyway. What I was trying to 
benchmark was the memory footprint during the parsing stage and after 
it, when only the resulting data should be residing in memory.