[Haskell-cafe] Aeson memory use

[Vanessa McHale]

Well, the problem with that is that in a lazy language, the "parsing
stage" may not be as clearly distinguished as you suggest. Simply
forcing the evaluation of all THUNKs is not what happens in the actual
program.


On 08/09/2018 07:57 AM, Lana Black wrote:
> On 08/08/2018 08:24 PM, Vanessa McHale wrote:
>> I'm not 100% sure it's applicable in your case, but perhaps trying
>> threadscope would be a good idea?
>>
>> Based on the heap profile, I'm guessing that most of the allocations are
>> due to slicing ByteStrings up. What happens when you try to read a map
>> of Integers?
>
> Parsing a 'Map Int Int' gives a similar picture, with slightly lower
> memory use due to the absence of [Text]. However, it is still in the
> hundreds of megabytes for a 10 megabyte file. I also noticed that Map
> itself contributes significantly to the memory footprint.
>
>> I'd also note that your example uses evaluate and performGC which isn't
>> entirely realistic and thus may not actually correspond to the
>> performance of the application in practice. I gather it *was* a problem
>> in practice but nonetheless it's always good to benchmark the right
>> thing.
>>
>
> Removing evaluate and force adds THUNKs to the heap profile, which is
> probably the wrong thing to benchmark anyway. What I was trying to
> benchmark was the memory footprint during the parsing stage and after
> it, when only the resulting data should be residing in memory.
> _______________________________________________
> Haskell-Cafe mailing list
> To (un)subscribe, modify options or view archives go to:
> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
> Only members subscribed via the mailman list are allowed to post.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: OpenPGP digital signature
URL: <http://mail.haskell.org/pipermail/haskell-cafe/attachments/20180809/00954343/attachment.sig>