[Haskell-cafe] MapReduce reverse engineered

[Alberto G. Corona]

2009/2/25 Alberto G. Corona <agocorona at gmail.com>

> And it would solve a lot of problem: scalability, system re-configuraition
> and installation: just by  adding or removing nodes at runtime.. heavy
>  numerical computations are also good candidates.
>
> 2009/2/25 Rick R <rick.richardson at gmail.com>
>
> I agree. A distributed database could be made as usable as a standard
>> RDBMS by offering an interface by which you supply a map/reduce pair
>> of functions and a list (range?) of keys.
>>
>> This could be easily implemented with a database such as Scalaris, in
>> which the Chord algorithm is responsible for placing and finding the
>> data among nodes.
>>
>> The user would interface with any node in the distributed database,
>> supplying a map and reduce function. It would distribute the map
>> function to nodes of its choosing (weighted by some metrics such as
>> idle cpu), retrieve the intermediate sets and run reduce if supplied.
>>
>>
>> 2009/2/25 Alberto G. Corona <agocorona at gmail.com>:
>> > Galchin,
>> >
>> > Maybe you are asking not only about remote execution, but also mobility
>> of
>> > code. This is a problem that is previous to mapReduce, since mapReduce
>> > assumes that all the code (and the data) is in place in the respective
>> > nodes. In fact, the distribution of resources in order to efficiently
>> use
>> > mapReduce is a design problem that the google people has done by hand.
>> > But  my intuition says that there are a general algorithm  for
>> distribution
>> > of  code, data, bandwidth and resources in general that  moves around at
>> > execution time to achieve better and better performance for a given grid
>> of
>> > nodes and for any task, for example, a mapReduce task. I would be very
>> > interesting to read something about this.
>> > I know that some efforts have been carried out the past , for example
>> mobile
>> > haskell
>> >
>> http://homepages.inf.ed.ac.uk/stg/workshops/TFP/book/DuBois/duboismhaskell/cameraready.pdf
>> > which is a first step for this goal but I this has been discontinued and
>> the
>> > source code is not available.
>> >
>> > 2009/2/25 Galchin, Vasili <vigalchin at gmail.com>
>> >>
>> >> Hello,
>> >>
>> >>      Here is an interesting paper of Google's MapReduce reverse
>> engineered
>> >> into Haskell. I apologize if already posted .....
>> >> http://www.cs.vu.nl/~ralf/MapReduce/
>> >>
>> >> Kind regards, Vasili
>> >>
>> >> _______________________________________________
>> >> Haskell-Cafe mailing list
>> >> Haskell-Cafe at haskell.org
>> >> http://www.haskell.org/mailman/listinfo/haskell-cafe
>> >>
>> >
>> >
>> > _______________________________________________
>> > Haskell-Cafe mailing list
>> > Haskell-Cafe at haskell.org
>> > http://www.haskell.org/mailman/listinfo/haskell-cafe
>> >
>> >
>>
>>
>>
>> --
>> We can't solve problems by using the same kind of thinking we used
>> when we created them.
>>    - A. Einstein
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.haskell.org/pipermail/haskell-cafe/attachments/20090225/a38a9a4f/attachment.htm