Extended periods of "waking up thread %d on cap %d"

[Ben Gamari]

Ben Gamari <bgamari.foss at gmail.com> writes:

> Recently I've been benchmarking my concurrent Gibbs sampler[1] on a
> largish multicore machine. I've been using GHC HEAD due to various
> GC-related fixes that are present. One thing that I've noticed in
> looking over the event logs is that there are large durations (tens of
> milliseconds) when HECs appear to be constantly bombarded with wake-ups
> from other capabilities.
>
> In these periods, the eventlog will be filled with blocks of messages
> such as this snippet from one benchmark run[2] (long repeated blocks
> marked with ellipses, a few irrelevant messages have been omitted yet
> not marked, see eventlog for full log),
>
>      28.320958s    HEC 7: running thread 293

I should note that this brings up what might be a related (perhaps more
significant from a performance perspective) issue. Notice that around
28.320s in eventlog [2], the benchmark began a new iteration (with a
corresponding lull in the CPU activity shown in the log). At this
point, a new batch of worker threads were created. Strangely, it seems
that the newly created threads begin execution in two groups: 
While HECs 0 and 2 through 16 begin immediately, HECs 1 and 17 through
23 all only begin at around 28.40s, after the "waking up thread 284 on
cap 7" storm subsides. Given this coincidence, it seems that these two
issues may be related.

Thoughts?

Cheers,

- Ben