[Haskell-cafe] Re: Re[8]: Parallel combinator, performance advice

[Neil Mitchell]

Hi Bulat,

>> Yes, this saves us adding a kill job to the queue, but requires an
>> extra MVar. I guess which one of these is to be preferred depends on
>> performance measures.
>
> i think that main problem with your last variant is that kill job
> added to the queue much later than real jobs. imagine the following
> scenario:
>
> 1. para used to add 100 jobs to the queue
> 2. while these jobs are executing, para in another thread adds another
> 100 jobs to the queue
>
> in my variant, first para exits just when its own jobs are completed.
> in your variant, exit job is added to the queue only after second
> batch of 100 jobs so we will wait until all these jobs are executed

Ah, you are indeed right. I forgot about the property of timeliness -
you want parallel_ to return as soon as it can.

Once you're using an MVar to do the killing, you might as well use the
same MVar to wait on, and avoid the semaphores, and avoid the bug with
negative semaphores: http://hackage.haskell.org/trac/ghc/ticket/3159

I've attached a revised implementation. With my benchmark it gives a
stack overflow:

n = 1000000

main = do
    r <- newIORef (0 :: Int)
    let incRef = atomicModifyIORef r (\a -> let a2 = a + 1 in a2 `seq` (a2,a))
    parallel_ $ replicate n $ incRef
    v <- readIORef r
    parallelStop
    print v

However, calling with that many items in a parallel queue is highly
unlikely, so I'll not worry (if anyone knows why it stack overflows, I
am curious). This parallel_ implementation has all the properties I
want, so I'm very happy with it.

As a side note, it's necessary to add parallelStop, to kill all the
threads - or you get thread blocked exceptions being raised. You
create a thread pool waiting on a queue as a CAF, but after the last
call to parallel_ finishes GHC realises the queue is unreachable. This
means that all the threads are blocked indefinitely, so GHC raises
exceptions. To solve this we call parallelStop to get all the threads
to die nicely.

Thanks

Neil
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Parallel3.hs
Type: text/x-haskell
Size: 2128 bytes
Desc: not available
Url : http://www.haskell.org/pipermail/haskell-cafe/attachments/20090408/465af920/Parallel3.bin