[Haskell-cafe] Parallel combinator, performance advice

[Neil Mitchell]

Hi,

I've written a parallel_ function, code attached. I'm looking for
criticism, suggestions etc on how to improve the performance and
fairness of this parallel construct. (If it turns out this construct
is already in a library somewhere, I'd be interested in that too!)

The problem I'm trying to solve is running system commands in
parallel. Importantly (unlike other Haskell parallel stuff) I'm not
expecting computationally heavy Haskell to be running in the threads,
and only want a maximum of n commands to fire at a time. The way I'm
trying to implement this is with a parallel_ function:

parallel_ :: [IO a] -> IO ()

The semantics are that after parallel_ returns each action will have
been executed exactly once. The implementation (attached) creates a
thread pool of numCapabililties-1 threads, each of which reads from a
task pool and attempts to do some useful work. I use an idempotent
function to ensure that all work is done at most one, and a sequence_
to ensure all work is done at least once.

Running a benchmark of issuing 1 million trivial tasks (create,
modify, read and IO ref) the version without any parallelism is really
fast (< 0.1 sec), and the version with parallelism is slow (> 10 sec).
This could be entirely due to space leaks etc when queueing many
tasks.

I'm useful for any thoughts people might have!

Thanks in advance,

Neil
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Parallel.hs
Type: application/octet-stream
Size: 1348 bytes
Desc: not available
Url : http://www.haskell.org/pipermail/haskell-cafe/attachments/20090407/252130a2/Parallel.obj