[Haskell-cafe] SMP parallelism increasing GC time dramatically

[Edward Z. Yang]

Ketil,

For your particular problem, unevaluated thunks should be easy
to check: dump a heap profile and look for a decreasing allocation
of thunks.

That being said, IntMap is spine strict, so that will all be evaluated,
and if your threads are accessing disjoint keys there should be no
contention. If there is, yes, threads will be blocking on evaluation,
I don't have a good sense for how slow that tends to be.  (Cache effects
may be swamping you.)

You may be interested in repa, if your maps are dense.

Edward

Excerpts from Ketil Malde's message of Wed Oct 05 17:00:11 -0400 2011:
> 
> I don't know if this is relevant to your problems, but I'm currently
> struggling to get some performance out of a parallel - or rather,
> concurrent - program.
> 
> Basically, the initial thread parses some data into an IntMap, and then
> multiple threads access this read-only to do the Real Work.
> 
> Now, there appears to be a lot of overhead incurred when using multiple
> threads, and I suspect that this is caused by the map storing
> unevaluated thunks, which then are forced by accesses by the worker
> threads.  Ideally, the evaluation should be performed in parallel, but
> perhaps there are issues (of synchronization, say) that makes this less
> performant?
> 
> -k