[Haskell-cafe] Space leak - help needed

[Justin Bailey]

On Thu, Mar 13, 2008 at 4:50 PM, Krzysztof Kościuszkiewicz
<k.kosciuszkiewicz at gmail.com> wrote:
>  Retainers are thunks or objects on stack that keep references to
>  live objects. All retainers of an object are called the object's
>  retainer set.  Now when one makes a profiling run, say with ./jobname
>  +RTS -p -hr, the graph refernces retainer sets from jobname.prof. My
>  understanding is that it is the total size of all objects retained by
>  retainer sets being plotted, correct?

Yes, all retainer sets are being profiled. However, you can FILTER the
retainer sets profiled to those containing certain cost-centres. This
is a key point because it allows you to "divide-and-conquer" when
tracking down a retainer leak. That is, if you filter to a certain
cost-centre and the retainer graph is flat, you know that cost-centre
is not involved. For example, if you have a cost-centre annotation
like {-# SCC "leaky" #-} in your code, you can filter the retainer set
like this:

  Leaky.exe +RTS -hr -hCleaky -RTS

Review the documentation for other options.

>
>  About decoding the sets from jobname.prof - for example in
>
>  > SET 2 = {<MAIN.SYSTEM>}
>  > SET 16 = {<Main.CAF>, <MAIN.SYSTEM>}
>  > SET 18 = {<MAIN.SYSTEM>, <Main.many1,Main.list,Main.expr,Main.CAF>}
>
>  {...} means it's a set, and <ccN,...,cc0> is the retainer cost centre
>  (ccN) and hierarchy of parent cost centres up to the "top level" (cc0)?
>
>  My understanding is that SET 18 above refers to objects that are
>  retained by exactly two specified cost centres, right?
>

The docs say

  "An object B retains object A if (i) B is a retainer object and (ii)
object A can be reached by recursively following pointers starting
from object B, but not meeting any other retainer objects on the way.
Each live object is retained by one or more retainer objects,
collectively called its retainer set ..."

That says to me that SET18 above is the set of all objects which are
retained by those two "call stacks", and only those call stacks. The
individual <..> items aren't "call stacks" but I think they refer to
where the retaining object (B in the paragraph) was itself retained,
so they are like call stacks. My intuition is very fuzzy here.

>  Finally, what is the MAIN.SYSTEM retainer?

I think that is "everything else" - any object created in the runtime
system that is not directly attributable to something being profiled.
Maybe it is objects from libraries that were not compiled with
profiling? I imagine objects created by the GHC primitives would fall
in this category too.

Since someone else found your space leak, does the retainer profiling
advice point to it? I'd like to know if it is actually accurate or
not! I've only applied it in some very limited situations.

Justin