[Haskell-cafe] monoid fold concurrently

[Viktor Dukhovni]

On Fri, Nov 15, 2019 at 09:29:58PM +0000, PICCA Frederic-Emmanuel wrote:

> > > Is the list is guaranteed non-empty?  Do you want to enforce that at the type
> > > level (compile time), or "fail" at runtime when it is?
> 
> runtime error

If there's an identity (empty) cube, then an empty list can perhaps
just map to that, but you could still throw an error.

> > > You should probably be a bit more explicit about what you mean by
> > > "concurrently".  Do you know in advance that the list length is sufficiently
> > > short to make it reasonable to immediately fork an async thread for each?
> 
> the list can have 100000 images of 1 million of pixels, and it is not that short per image.

This suggests that processing them should not happen all at once, but rather
you'd load more images as idle task slots become available.

This suggests use of 'mapReduce' from:

    https://hackage.haskell.org/package/async-pool-0.9.0.2/docs/Control-Concurrent-Async-Pool.html

> > Is it actually a Monoid (has an identity), or only a Semigroup?
> 
> I can have a identiy which is an empty cube.

This is good, since mapReduce wants a Monoid.

> computing the cube is constant in time for a given image size.
> merging cube is slower and slower with time, since the cube grows from merge to merge.

It looks like mapReduce also reduces pairs in parallel, and discards
the inputs.  It looks to be order preserving, so you don't even need
commutativity.

I've not used this module myself, please post a summary of your
experience.

-- 
    Viktor.