[Haskell-cafe] MapReduce reverse engineered

[Rick R]

I agree. A distributed database could be made as usable as a standard
RDBMS by offering an interface by which you supply a map/reduce pair
of functions and a list (range?) of keys.

This could be easily implemented with a database such as Scalaris, in
which the Chord algorithm is responsible for placing and finding the
data among nodes.

The user would interface with any node in the distributed database,
supplying a map and reduce function. It would distribute the map
function to nodes of its choosing (weighted by some metrics such as
idle cpu), retrieve the intermediate sets and run reduce if supplied.


2009/2/25 Alberto G. Corona <agocorona at gmail.com>:
> Galchin,
>
> Maybe you are asking not only about remote execution, but also mobility of
> code. This is a problem that is previous to mapReduce, since mapReduce
> assumes that all the code (and the data) is in place in the respective
> nodes. In fact, the distribution of resources in order to efficiently use
> mapReduce is a design problem that the google people has done by hand.
> But  my intuition says that there are a general algorithm  for distribution
> of  code, data, bandwidth and resources in general that  moves around at
> execution time to achieve better and better performance for a given grid of
> nodes and for any task, for example, a mapReduce task. I would be very
> interesting to read something about this.
> I know that some efforts have been carried out the past , for example mobile
> haskell
> http://homepages.inf.ed.ac.uk/stg/workshops/TFP/book/DuBois/duboismhaskell/cameraready.pdf
> which is a first step for this goal but I this has been discontinued and the
> source code is not available.
>
> 2009/2/25 Galchin, Vasili <vigalchin at gmail.com>
>>
>> Hello,
>>
>>      Here is an interesting paper of Google's MapReduce reverse engineered
>> into Haskell. I apologize if already posted .....
>> http://www.cs.vu.nl/~ralf/MapReduce/
>>
>> Kind regards, Vasili
>>
>> _______________________________________________
>> Haskell-Cafe mailing list
>> Haskell-Cafe at haskell.org
>> http://www.haskell.org/mailman/listinfo/haskell-cafe
>>
>
>
> _______________________________________________
> Haskell-Cafe mailing list
> Haskell-Cafe at haskell.org
> http://www.haskell.org/mailman/listinfo/haskell-cafe
>
>



-- 
We can't solve problems by using the same kind of thinking we used
when we created them.
    - A. Einstein