[Haskell-cafe] MapReduce reverse engineered
Alberto G. Corona
agocorona at gmail.com
Wed Feb 25 13:16:35 EST 2009
2009/2/25 Alberto G. Corona <agocorona at gmail.com>
> And it would solve a lot of problem: scalability, system re-configuraition
> and installation: just by adding or removing nodes at runtime.. heavy
> numerical computations are also good candidates.
> 2009/2/25 Rick R <rick.richardson at gmail.com>
> I agree. A distributed database could be made as usable as a standard
>> RDBMS by offering an interface by which you supply a map/reduce pair
>> of functions and a list (range?) of keys.
>> This could be easily implemented with a database such as Scalaris, in
>> which the Chord algorithm is responsible for placing and finding the
>> data among nodes.
>> The user would interface with any node in the distributed database,
>> supplying a map and reduce function. It would distribute the map
>> function to nodes of its choosing (weighted by some metrics such as
>> idle cpu), retrieve the intermediate sets and run reduce if supplied.
>> 2009/2/25 Alberto G. Corona <agocorona at gmail.com>:
>> > Galchin,
>> > Maybe you are asking not only about remote execution, but also mobility
>> > code. This is a problem that is previous to mapReduce, since mapReduce
>> > assumes that all the code (and the data) is in place in the respective
>> > nodes. In fact, the distribution of resources in order to efficiently
>> > mapReduce is a design problem that the google people has done by hand.
>> > But my intuition says that there are a general algorithm for
>> > of code, data, bandwidth and resources in general that moves around at
>> > execution time to achieve better and better performance for a given grid
>> > nodes and for any task, for example, a mapReduce task. I would be very
>> > interesting to read something about this.
>> > I know that some efforts have been carried out the past , for example
>> > haskell
>> > which is a first step for this goal but I this has been discontinued and
>> > source code is not available.
>> > 2009/2/25 Galchin, Vasili <vigalchin at gmail.com>
>> >> Hello,
>> >> Here is an interesting paper of Google's MapReduce reverse
>> >> into Haskell. I apologize if already posted .....
>> >> http://www.cs.vu.nl/~ralf/MapReduce/
>> >> Kind regards, Vasili
>> >> _______________________________________________
>> >> Haskell-Cafe mailing list
>> >> Haskell-Cafe at haskell.org
>> >> http://www.haskell.org/mailman/listinfo/haskell-cafe
>> > _______________________________________________
>> > Haskell-Cafe mailing list
>> > Haskell-Cafe at haskell.org
>> > http://www.haskell.org/mailman/listinfo/haskell-cafe
>> We can't solve problems by using the same kind of thinking we used
>> when we created them.
>> - A. Einstein
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Haskell-Cafe