[Haskell-cafe] a parallel mapM?

[Greg Fitzgerald]

I'm new to concurrent programming in Haskell.  I'm looking for a
drop-in replacement for 'mapM' to parallelize a set of independent IO
operations.  I hoped 'mapConcurrently' might be it, but I need
something that will only spawn as many threads as I have CPUs
available [1].

I also tried Control.Parallel.Strategies [2].  While that route works,
I had to use unsafePerformIO.  Considering that IO is for sequencing
effects and my IO operation doesn't cause any side-effects (besides
hogging a file handle), is this a proper use of unsafePerformIO?


Attempt 1
--------------

import System.Process(readProcess)
import Control.Concurrent.Async(mapConcurrently)

main :: IO [String]
main = mapConcurrently (\n -> readProcess "echo" ["test: " ++ show n]
"") [0..1000]


$ ghc --version
The Glorious Glasgow Haskell Compilation System, version 7.6.1

$ runghc test.hs
test.hs: runInteractiveProcess: pipe: Too many open files
test.hs: runInteractiveProcess: pipe: Too many open files
test.hs: runInteractiveProcess: pipe: Too many open files
test.hs: runInteractiveProcess: pipe: Too many open files
test.hs: runInteractiveProcess: pipe: Too many open files
test.hs: runInteractiveProcess: pipe: Too many open files
test.hs: runInteractiveProcess: pipe: Too many open files
test.hs: runInteractiveProcess: pipe: Too many open files
test.hs: runInteractiveProcess: pipe: Too many open files
test.hs: echo: createProcess: resource exhausted (Too many open files)


Attempt 2
--------------

import System.Process(readProcess)
import Control.Parallel.Strategies(parMap, rpar)
import System.IO.Unsafe(unsafePerformIO)

main :: IO [String]
main = myMapConcurrently (\n -> readProcess "echo" ["test: " ++ show
n] "") [0..1000]
  where
    myMapConcurrently f = return . parMap rpar (unsafePerformIO . f)

$ runghc test.hs > /dev/null && echo Success
Success


Thanks,
Greg