Residency profiles

[Ömer Sinan Ağacan]

Hi,

 > I think what we want is a way to trigger GC at very regular intervals, after
 > (say) each 10kbytes or 100kbytes or 1Mbyte  of allocation.  That might be
 > expensive, but we’d get reproducible results.

If we could fix the nursery size to 10kb that'd trigger a GC in every 10kb of
allocation (you could still allocate large objects as those are not allocated in
the nursery, but perhaps that's not a problem in your benchmarks). Then by
setting -G1 you could turn all GCs to major GCs (because first generation is
always collected). Note that because each capability has its own nursery you may
want to set the nursery size to alloc_per_gc / num_of_caps if you need more than
one capability.

 > I don’t think that is possible right now – see the ticket – but it would be
 > easy enough to do wouldn’t it?  Just give only 10k or 100k or 1M to the
 > allocator when setting it running again.

Right. A parameter for fixing the nursery size would be easy to implement, I
think. Just a new flag, then in GC.c:resize_nursery() use the flag as the
nursery size.

"Max. residency" is really hard to measure (need to do very frequent GCs),
perhaps a better question to ask is "residency when the program is in state S".
This is also hard to measure if your program is threaded or have other
non-determinism, but this lets you decide when to measure residency. Currently
we can't tell the GC to print residency stats, but perhaps we could implement a
variant of `performGC` that prints residency after the GC. So in your program
you could add `performGCPrintStats` after every iteration or step etc. Not sure
how useful this would be, but just an idea..

On 6.12.2018 13:09, Simon Peyton Jones wrote:
> Simon, Ben, Omer
> 
> As you’ll see in comments 55-72 of https://ghc.haskell.org/trac/ghc/ticket/9476, 
> Sebastian has been a bit flummoxed by the task of measure residency profiles; 
> that is, how much data is truly live during execution.
> 
> A major GC measures that, but we are vulnerable to exactly when it happens (even 
> with -G1) and that can lead to irreproducible results.
> 
> I think what we want is a way to trigger GC at very regular intervals, after 
> (say) each 10kbytes or 100kbytes or 1Mbyte  of allocation.  That might be 
> expensive, but we’d get reproducible results.
> 
> I don’t think that is possible right now – see the ticket – but it would be easy 
> enough to do wouldn’t it?  Just give only 10k or 100k or 1M to the allocator 
> when setting it running again.
> 
> Would you consider this?  Or are we just missing something obvious?
> 
> Needless to say, we want to do all this with full optimisation on, no 
> cost-centre profiling.
> 
> Thanks
> 
> Simon
> 

-- 
Ömer Sinan Ağacan, Haskell Consultant
Well-Typed LLP, http://www.well-typed.com

Registered in England & Wales, OC335890
118 Wymering Mansions, Wymering Road, London W9 2NF, England