Profiling burn-rate

[Sengan]

"Julian Seward (Intl Vendor)" wrote:
> 
> | Is there any way to profile the "burn-rate" of a Haskell program?
> |
> | Profiling as I understand it tells you what the "live" information
> | on the heap is. It doesn't tell you what garbage collector has just
> | freed. So, if a function were generating tons of intermediate values
> | which it then immediately consumed, you wouldn't see it on
> | the profile.
> | Yet this is a place where rewriting the code or adding
> | strictness could
> | help reduce time spent garbage collecting.
> 
> At least in GHC, and, I suspect, for any system with a good
> generational GC, the above may not really hold.  A good Gen GC
> should get rid of short-lived junk very cheaply, since that's
> what it's designed to do.  My experience is that a more effective
> way to reduce GC time is to reduce the residency.  In other words
> it's the area bounded by the space-time curve that counts, IMO.
> 
> J

I understand that reducing residency will help: if 95% of the heap
is occupied by high residency stuff, then you'll run out of heap much
more often than if it's empty. And that has been one of the ways I
have improved performance to date.

However it still seems to me that short-lived junk has 2 problems:
a) It is created    (takes time)
b) It is destroyed  (takes more time).
And it seems to me that if I knew what functions create short term junk
I might find ways of replacing them with functions that don't.

For instance, in a graph searching algorithm, I had a scenario which boils
down to: concatMap f  where  f is a combination of functions that end up being
(\x -> if condition x then [] else [g x]). When I wrote it, I thought that most
of the time I would get [g x] but in fact, most of the time I get []. There turned
out to be a completely different way of generating the list, which took much less
processing. Had I had a means of measuring profiling burn-rate, I'm guessing that
it would have brought this to my attention earlier. I only found it because of a
guess that I checked with Ghood.

Sengan