[Haskell-cafe] Re: Parallel combinator, performance advice
ChrisK
haskell at list.mightyreason.com
Tue Apr 7 09:48:45 EDT 2009
Neil Mitchell wrote:
> Sorry, accidentally sent before finishing the response! I also noticed
> you sent this directly to me, not to -cafe, was that intentional?
The mail/news gateway makes it look like that, but I also sent to the mailing list.
> You mean something like:
>
> parallel_ xs =
> sem <- createSemapore (length xs)
> enqueue [x >> signalSemapore sem | x <- xs]
> waitOnSemaphore sem
>
> I thought of something like this, but then the thread that called
> parallel_ is blocked, which means if you fire off N threads you only
> get N-1 executing. If you have nested calls to parallel, then you end
> up with thread exhaustion. Is there a way to avoid that problem?
>
> Thanks
>
> Neil
Your parallel_ does not return until all operations are finished.
> parallel_ (x:xs) = do
> ys <- mapM idempotent xs
> mapM_ addParallel ys
> sequence_ $ x : reverse ys
By the way, there is no obvious reason to insert "reverse" there.
What I meant was something like:
>
> para [] = return ()
> para [x] = x
> para xs = do
> q <- newQSemN 0
> let wrap x = finally x (signalQSemN q 1)
> go [y] n = wrap x >> waitQSemN q (succ n)
> go (y:ys) n = addParallel (wrap y) >> go ys $! succ n
> go xs 0
This is nearly identical to your code, and avoid creating the MVar for each
operation. I use "finally" to ensure the count is correct, but if a worker
threads dies then bas things will happen. You can replace finally with (>>) if
speed is important.
This is also lazy since the length of the list is not forced early.
More information about the Haskell-Cafe
mailing list