[Haskell-beginners] Parallelism?

[Simon Marlow]

On 02/12/2011 00:08, Michael Craig wrote:
> Thanks, I'll keep that in mind.
>
> I wrote a better example of what I'm trying to do:
> https://gist.github.com/1420742 It runs worse with multiple threads than
> it does with one (both time-wise and memory-wise), I think due to the
> bug Simon described in that commit.

To be clear, the fix was for the problem where your program ran much 
slower with -threaded than without.  It probably won't have much effect 
on scaling.

In the program you linked above, I'm not terribly surprised if it 
doesn't scale well - it is basically a communication benchmark, and 
those perform best when running both threads on a single core.  In fact, 
I would go so far as to pin them both to the same core with forkOnIO.

Perhaps you might expect that, since the channel is unbounded, the 
writer thread should be able to produce data in batches that is then 
slurped up by the reader in batches.  Sure - but when running on two 
cores the data has to be moved from the cache of one core to the other, 
and that's a killer.  Much better to keep it in the cache of one core.

> But that bug aside, is it possible to write a multithreaded application
> in Haskell that sends large amounts of data from one thread to the other
> without getting crushed by GC? I've looked into the garbage collector a
> bit, and this seems problematic.

I don't think it's so much the GC as it is the cost of moving data 
across the memory bus.  Communication should be nice and fast if you 
keep it all on one core.  If I'm wrong please send me the code and I'll 
look into it!

Cheers,
	Simon



> Mike S Craig
>
>
> On Thu, Dec 1, 2011 at 2:38 PM, Edward Z. Yang <ezyang at mit.edu
> <mailto:ezyang at mit.edu>> wrote:
>
>     OK. A common mistake when using channels is forgetting to make sure
>     all of the informaiton is fully evaluated in the worker thread,
>     which then
>     causes the writer thread to spend all its time evaluating thunks.
>
>     Edward
>
>     Excerpts from Michael Craig's message of Thu Dec 01 13:17:57 -0500
>     2011 <tel:57%20-0500%202011>:
>      > Excellent! Glad this has been sorted out upstream.
>      >
>      > Edward, to answer your question regarding blocking database
>     calls: I'm
>      > using mongoDB to log events that come into a WAI webapp. The
>     writes to
>      > mongo are blocking, so I'd like to run them in parallel with the
>     webapp.
>      > The webapp would push the data into a Chan and the mongo writer
>     would read
>      > from the Chan and make the writes sequentially (using the Chan as
>     a FIFO
>      > between parallel threads). This would allow for request rates to
>      > temporarily rise above mongo's write rate (of course with an expanded
>      > memory footprint during those bursts).
>      >
>      > Mike S Craig
>      >
>      >
>      > On Thu, Dec 1, 2011 at 12:10 PM, Felipe Almeida Lessa <
>      > felipe.lessa at gmail.com <mailto:felipe.lessa at gmail.com>> wrote:
>      >
>      > > On Thu, Dec 1, 2011 at 2:40 PM, Edward Z. Yang <ezyang at mit.edu
>     <mailto:ezyang at mit.edu>> wrote:
>      > > > Simon Marlow investigated, and we got this patch out:
>      > >
>      > > Nice work, guys!  Hope it gets included in the glourious GHC
>     7.4 =D.
>      > >
>      > > Cheers,
>      > >
>      > > --
>      > > Felipe.
>      > >
>
>
>
>