[Haskell-cafe] Parallel combinator, performance advice

[Sebastian Sylvan]

This is a random idea, that's probably not going to work, but I don't have a
way of testing it so I'll just post it!
How about using unsafeInterleaveIO to get a lazy suspension of the result of
each action, and then using par to spark off each of them? If that works you
can reuse the existing task-parallel system of GHC to do the heavily lifting
for you, instead of having to write your own.

On Tue, Apr 7, 2009 at 11:25 AM, Neil Mitchell <ndmitchell at gmail.com> wrote:

> Hi,
>
> I've written a parallel_ function, code attached. I'm looking for
> criticism, suggestions etc on how to improve the performance and
> fairness of this parallel construct. (If it turns out this construct
> is already in a library somewhere, I'd be interested in that too!)
>
> The problem I'm trying to solve is running system commands in
> parallel. Importantly (unlike other Haskell parallel stuff) I'm not
> expecting computationally heavy Haskell to be running in the threads,
> and only want a maximum of n commands to fire at a time. The way I'm
> trying to implement this is with a parallel_ function:
>
> parallel_ :: [IO a] -> IO ()
>
> The semantics are that after parallel_ returns each action will have
> been executed exactly once. The implementation (attached) creates a
> thread pool of numCapabililties-1 threads, each of which reads from a
> task pool and attempts to do some useful work. I use an idempotent
> function to ensure that all work is done at most one, and a sequence_
> to ensure all work is done at least once.
>
> Running a benchmark of issuing 1 million trivial tasks (create,
> modify, read and IO ref) the version without any parallelism is really
> fast (< 0.1 sec), and the version with parallelism is slow (> 10 sec).
> This could be entirely due to space leaks etc when queueing many
> tasks.
>
> I'm useful for any thoughts people might have!
>
> Thanks in advance,
>
> Neil
>
> _______________________________________________
> Haskell-Cafe mailing list
> Haskell-Cafe at haskell.org
> http://www.haskell.org/mailman/listinfo/haskell-cafe
>
>


-- 
Sebastian Sylvan
+44(0)7857-300802
UIN: 44640862
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.haskell.org/pipermail/haskell-cafe/attachments/20090407/6b18f0c2/attachment.htm