File APIs for Java Developers
Manipulate DOC, XLS, PPT, PDF and many others from your application.
http://aspose.com/file-tools
The moose likes Hadoop and the fly likes MapReduce model used in Hadoop & google Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Databases » Hadoop
Bookmark "MapReduce model used in Hadoop & google" Watch "MapReduce model used in Hadoop & google" New topic
Author

MapReduce model used in Hadoop & google

Kuladip Yadav
Ranch Hand

Joined: Jul 30, 2008
Posts: 162

Hi Chuck,

Does the Google uses same MapReduce model as in Hadoop ?
And have you discussed this topic in your book ?

Thanks.
Tibi Kiss
Ranch Hand

Joined: Jun 11, 2009
Posts: 47
Hadoop's MapReduce model it is very similar with Google's MapReduce model. If we consider what the creators of the Hadoop tell us, that they started to design Hadoop after reading the GFS and MapReduce papers published by Google, it means that is the same model.

Of course there are some differences. Not so far ago I read Data-Intensive Text processing with MapReduce book written by Jimmy Lin and Chris Dyer, in that book I often saw mentioned the small differences between Hadoop and Google's MapReduce implementation.

I am asking the author of Hadoop in Action, if that book also covers such comparisons or mentions differences on the mode Google's MapReduce implementation was designed compared to Hadoop?
Kuladip Yadav
Ranch Hand

Joined: Jul 30, 2008
Posts: 162

Thanks Tibi for your reply.

But can anybody campare both of these model?

Thanks.
Chuck Lam
author
Greenhorn

Joined: Aug 09, 2010
Posts: 12
Hadoop is based on a couple research papers published by Google explaining Google's data processing model. So the *conceptual model* can be considered the same. Of course, the details are very different.

To start out, Google's MapReduce programs are generally written in C/C++, while Hadoop's are generally Java-based. Given that both models have evolved separately over the years to target different communities, it shouldn't be surprising that the details are very different. Having said that, Google engineers I've talked to claim that learning Hadoop is relatively easy for them.
Tibi Kiss
Ranch Hand

Joined: Jun 11, 2009
Posts: 47
Chuck Lam wrote:....To start out, Google's MapReduce programs are generally written in C/C++, while Hadoop's are generally Java-based. ....

I was expecting that Google also using java implementation.
Thanks for the information!
Kuladip Yadav
Ranch Hand

Joined: Jul 30, 2008
Posts: 162

Thanks Chuck for your information
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: MapReduce model used in Hadoop & google