[Haskell-cafe] Re: Random matrices

[Donald Bruce Stewart]

Chad.Scherrer:
>  
> > Ah ha. That'll do.
> > Lesson: avoid hidden space leaks in monads.
> 
> Hmm, I'm still missing something. It seems a good lesson, but
> practically speaking, it doesn't help me any more than saying "write
> efficient programs". What could I have looked for in the original code
> to predict it may be leaky? Is there a way of thinking about this that
> can help me stop hitting space leaks like this?

Normally, you can just profile the code revealing the function that's
causing trouble, and then usually the leak is obvious (at least once
you've done this a couple of times). Practice helps, as does an
understanding of the execution model (see the Spineless Tagless
G-machine paper). You really don't know what to look for, I guess,
unless you know what `let' versus `case' does at runtime.

Here however, it was the `hidden' allocations in the monad that were to
blame, so it took a bit more thinking to realise the problem.  Profiling
told us what function was misbehaving, but glancing at the function
didn't reveal anything obvious (except we should have realised that
lazily threading StdGen state was going to be a problem, with
hindsight).

The fact that there is this hidden allocation when you use a state monad
is a point to keep in mind - allocation where you don't write the `let'
yourself. Have a look at the definition of the State monad to see how
the state is actually threaded.

There's an interesting comment in the strict ST monad code,
fptools/libraries/base/GHC/ST.lhs:
    > By default the monad is strict; too many people got bitten by
    > space leaks when it was lazy.
Which is what happenened here, I think. For repeated, heavy calls like
this, maybe ST (or even IORefs in IO) are more your friends, than State.

But, as I say, I think the best way is to get an understanding of the
execution model, and get some experience squashing leaks, as they tend
to be of very similar form usually, so practice helps.

> BTW, I really do appreciate the help. I've been amazed at the level of
> effort put forth by the Haskell community as a whole to help out
> newcomers.

No worries!

-- Don