[Haskell-cafe] Haskell and Big Data

[Flavio Villanustre]

J-C,

I've been trying to port Haskell to the Open Source HPCC Systems platform (
http://hpccsystems.com/) over the last couple of months, but been swamped
by work and personal stuff, so I didn't move too far. In case you don't
know much about it, this is a Big Data platform originally designed and
built by LexisNexis (Reed Elsevier) about 15 years ago and released under
an Apache 2.0 license a couple of years ago. This platform continues to be
actively developed and extended and leverages clusters of nodes with share
nothing architectures and local storage, over IP based networks.

It presents several nice synergies with Haskell: the platform is coded in
C++ (as opposed to Java) and has a high level dataflow programming language
called ECL, which is compiled to C++, and already allows for embedding C++,
Java, Python, Javascript and R, so adding Haskell using any of those
existing examples should be fairly trivial and could be probably done in
just a few hours worth of work (see python, for example:
https://github.com/hpcc-systems/HPCC-Platform/blob/master/plugins/pyembed/pyembed.cpp).
Since ECL advocates non-strictness through purity too, with pure Haskell
code the ECL compiler should be fairly free to distribute and parallelize
Haskell code quite efficiently across the entire cluster. Another
interesting aspect of the HPCC Systems platform is that it has a
programmable distributed real-time data retrieval component called Roxie
(programmable in ECL too), so this same work would also give Haskellers a
distributed system for real-time large scale data delivery.

I'm posting this response here, with the hope that someone may be willing
to help out with this integration effort too.

Kind regards,

Flavio

Flavio Villanustre


On Wed, Dec 18, 2013 at 2:41 AM, jean-christophe mincke <
jeanchristophe.mincke at gmail.com> wrote:

> Hello Cafe,
>
> Big Data is a bit trendy these days.
>
> Does anybody know about plans to develop an Haskell eco-system in that
> domain?
> I.e tools such as Storm or Spark (possibly on top of Cloud Haskell) or, at
> least, bindings to tools which exist in other languages.
>
> Thank you
>
> Regards
>
> J-C
>
> _______________________________________________
> Haskell-Cafe mailing list
> Haskell-Cafe at haskell.org
> http://www.haskell.org/mailman/listinfo/haskell-cafe
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/haskell-cafe/attachments/20131218/491f0ed9/attachment.html>