[Haskell-cafe] Large, numerical calculations in Haskell

[Emil Hedevang]

Hi Haskell Cafe,

I need to perform very large numerical computations requiring tens of GB of
memory. The computations consist essentially of generation of random
numbers and discrete convolutions of large arrays of random numbers with
somewhat smaller kernel arrays. Should I use Haskell and call some C
libraries when necessary? Should I just write everything in C? Or should I
use e.g. Chapel (chapel.cray.com)?

To be more specific, a very large array X is filled with random numbers
drawn independently from some distribution. Then, the discrete convolution
Y = K * X of X with a somewhat smaller kernel array K is calculated.

Typical sizes of X and K could be 1000 x 1000 x 1000 and 100 x 100 x 100,
respectively. The storage of X alone requires 8 GB of memory. Luckily I
have recently been blessed with a well-equipped workstation with 128 GB RAM
and two 8-core Xeon E5-2650 running at 2.0 GHz.

I see immediate opportunities for parallelism. Firstly, parallel random
number generators could be employed to populate the array X (e.g. SPRNG).
Secondly, since the size of the kernel K is much smaller than the size of
the array X, the discrete convolution can split into several smaller
convolutions (see e.g. Kim and Strintzis (1980)). Finally, the discrete
convolutions can be computed efficiently using Fourier methods (using e.g.
FFTW).

It seems fairly straightforward to implement it all in C, perhaps with some
OpenMP pragmas scattered around in the code, since the heavy numerics would
be performed in preexisting libraries (SPRNG and FFTW),

I would like to use Haskell, since I like Haskell and find it tremendously
fascinating, but I will not use it for the sake of just using it. I am
willing to pay a small price in terms of performance, say, twice the run
time, if I in turn will get more convenience and less suffering while
coding. Therefore, would you claim it to be feasible to use Haskell? I
suppose it very much depends on the quality of the Haskell FFI bindings to
parallel random number generators and fast Fourier transforms.
Unfortunately, I find it difficult to assess these qualities.

Best regards,
Emil Hedevang
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/haskell-cafe/attachments/20121211/743d76e6/attachment.htm>