[Haskell-cafe] Parallel combinator, performance advice
sebastian.sylvan at gmail.com
Tue Apr 7 11:02:55 EDT 2009
This is a random idea, that's probably not going to work, but I don't have a
way of testing it so I'll just post it!
How about using unsafeInterleaveIO to get a lazy suspension of the result of
each action, and then using par to spark off each of them? If that works you
can reuse the existing task-parallel system of GHC to do the heavily lifting
for you, instead of having to write your own.
On Tue, Apr 7, 2009 at 11:25 AM, Neil Mitchell <ndmitchell at gmail.com> wrote:
> I've written a parallel_ function, code attached. I'm looking for
> criticism, suggestions etc on how to improve the performance and
> fairness of this parallel construct. (If it turns out this construct
> is already in a library somewhere, I'd be interested in that too!)
> The problem I'm trying to solve is running system commands in
> parallel. Importantly (unlike other Haskell parallel stuff) I'm not
> expecting computationally heavy Haskell to be running in the threads,
> and only want a maximum of n commands to fire at a time. The way I'm
> trying to implement this is with a parallel_ function:
> parallel_ :: [IO a] -> IO ()
> The semantics are that after parallel_ returns each action will have
> been executed exactly once. The implementation (attached) creates a
> thread pool of numCapabililties-1 threads, each of which reads from a
> task pool and attempts to do some useful work. I use an idempotent
> function to ensure that all work is done at most one, and a sequence_
> to ensure all work is done at least once.
> Running a benchmark of issuing 1 million trivial tasks (create,
> modify, read and IO ref) the version without any parallelism is really
> fast (< 0.1 sec), and the version with parallelism is slow (> 10 sec).
> This could be entirely due to space leaks etc when queueing many
> I'm useful for any thoughts people might have!
> Thanks in advance,
> Haskell-Cafe mailing list
> Haskell-Cafe at haskell.org
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Haskell-Cafe