heap profiling

[Simon Marlow]

On 16/06/2010 14:04, Jesper Louis Andersen wrote:

> Finally, you can tune the eden/nursery generation by providing +RTS
> -A5m for instance. The default is 0.5 megabytes which I think is way
> too low for most work. Increasing it blindly will hurt though as you
> will hit L1/L2 cache limits and get worse performance.

Not sure I follow here - you say that 0.5MB is too low, and yet 
increasing it will hurt performance (I agree with you on the last point: 
in my experience most of the time increasing it does tend to make things 
worse).  OTOH there are programs where the 0.5MB default really hurts - 
the binary-trees benchmark on the shootout, for example.

If we could identify a useful set of heuristics to decide when it's a 
good idea to increase the nursery size, that would help.  I think the 
only way to get a real handle on this is to put together a comprehensive 
benchmark suite and test lots of different programs with different heap 
requirements, and on lots of different machines with different cache sizes.

Incedentally, GHC HEAD has a new flag +RTS -H, which increases the 
nursery size but without increasing the overall memory use (basically it 
uses the space left over after a copying major collection for the 
nursery).  This sometimes helps with programs that have large heaps, but 
not always.

Cheers,
	Simon


>
> My application has had problems with both retainer profiling and
> (especially) biographical profiling - but whenever I try to make a
> watered-down reproducible example the problem disappears. If you do
> manage the get a reproducible bug going, I am interested in that
> ticket.
>
>> So, the main question:
>>
>> I have a program that runs some computation in a monad stack before
>> extracting the final result, a list, and passing it on.  When run
>> under the heap profiler, there's a huge spike in that phase, which I
>> think should be mostly bogus usage, since the final output is so
>> relatively small.  When I run under -hb I see big bands for LAG and
>> DRAG.
>
> I would look at retainers. Or if you hold on to things you don't use
> anymore rather than dropping the reference. In general, the heap
> profiler is the way to go for removing space leak problems -
>