Massive slowdown in mwc-random after switching to use of primitive package

[Dan Doel]

On Saturday 10 July 2010 2:09:48 pm Bryan O'Sullivan wrote:
> Recently, I switched the mwc-random package (
> http://hackage.haskell.org/package/mwc-random) over from running in the ST
> monad to using your primitive package. I didn't notice initially, but this
> caused a huge performance regression.

You're using GHC 6.12.x presumably? There are known performance problems with 
using abstract PrimMonads in that version (and, actually, just using IO as 
well). The problem, as far as I (and Roman, I believe) have been able to 
ascertain, is a lack of some sort of dictionary inlining/specialization. So, 
when you use PrimMonad or IO, You get repeated casts between the state types. 
In the IO case for instance, checking the core tends to reveal lots of casting 
between RealWorld and PrimState IO, despite the fact that those should be 
identical. Whether this is directly responsible, or just prevents other 
optimizations from firing, I'm not totally sure.

For some reason, ST isn't affected, possibly because it holds its state thread 
type in a parameter. That means one workaround is to keep all your 
computations in ST, and as long as they aren't inlined into an stToIO or 
something, which would trigger the problem by specializing the code to ST 
RealWorld, you should have decent performance.

The good news is that, last I checked, this isn't a problem in HEAD. The 
optimizer does better there somehow. The bad news is that 6.14 isn't out yet, 
so you probably have to wait for a real fix.

-- Dan