heap profiling

[Evan Laforge]

On Sat, Jun 19, 2010 at 8:46 AM, Claus Reinke <claus.reinke at talk21.com> wrote:
>> I put the simple version at
>> http://hpaste.org/fastcgi/hpaste.fcgi/view?id=26329#a26329
>>
>> This one displays much better performance with DList + Writer.Strict
>> than List + StrictWriter so I guess it's not too surprising.  However,
>> *something* is still generating a lot of lag
>
> Some lag is to be expected: logging is about creating
> and recording data for later use. You'd need  to annotate

Indeed, though I don't think this is the case, because I get lots of
lag even when no logs are written.  I solved the stack overflow
problem with state by replacing 'modify' with a stricter '!x <- get;
put $! (x+1)'.  However, the results from profiling logging continue
to be mysterious.

The fastest is using a strict StateT, prepending with 'modify (msg:)'
and reversing the list on run.  Oddly, making *this* modify strict
results in a near 2x slowdown, perhaps due to a 27.7% productivity
instead of a 31.8% productivity (though both are terrible, and I still
don't know why all the garbage is being produced).  Using Simon's
AppendList (my implementation is based on OrdList from ghc), it's
slightly slower than the strict modify version.  Apparently reversing
a 4194304 element list is still faster than using a custom datatype.

My conclusion is that my contrived tests are not useful because the
whole point is to get a controlled situation by simplifying it, but
even the simple version displays mysterious behaviour.  I should go
back to poking at the production version.  Even if I don't understand
the changes I'm making, at least when I get a speedup I'm more sure it
translates to a real speed up in the application.