[Haskell-cafe] monoid fold concurrently
frederic-emmanuel.picca at synchrotron-soleil.fr
Fri Nov 15 20:20:44 UTC 2019
> Is the list is guaranteed non-empty? Do you want to enforce that at the type
> level (compile time), or "fail" at runtime when it is?
> You should probably be a bit more explicit about what you mean by
> "concurrently". Do you know in advance that the list length is sufficiently
> short to make it reasonably to immediately fork an async thread for each?
> Also, do you want the outputs to folded in list order, or in any order (e.g.
> roughly in order of IO action completion)?
The real probleme, is:
I have a bunch of hdf5 files which contain a stack of image and other metadata's.
for each image an associated metadatas, I can create a cube (3D array), whcih is the binning in the 3D space
for each image of the stack
read image + metadata -> transformation in 3D space -> binning.
binning -> binning -> binning (this is the monoid), since this is pure computation, I can use unsafe IO to create the merge function.
In my case, I want to distribute all this on all my core.
each core can do in which ever order a merge of the binning until I have only one binning.
[a1, a2, a3, a4]
core1: a1 + a2 -> a12
core2: a3 + a4 -> a34
first core available, a12 + a34 -> a1234
Is it clearer ?
More information about the Haskell-Cafe