This week's giveaway is in the Android forum.
We're giving away four copies of Android Security Essentials Live Lessons and have Godfrey Nolan on-line!
See this thread for details.
The moose likes Hadoop and the fly likes MapReduce model used in Hadoop & google Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of Android Security Essentials Live Lessons this week in the Android forum!
JavaRanch » Java Forums » Databases » Hadoop
Bookmark "MapReduce model used in Hadoop & google" Watch "MapReduce model used in Hadoop & google" New topic
Author

MapReduce model used in Hadoop & google

Kuladip Yadav
Ranch Hand

Joined: Jul 30, 2008
Posts: 162

Hi Chuck,

Does the Google uses same MapReduce model as in Hadoop ?
And have you discussed this topic in your book ?

Thanks.
Tibi Kiss
Ranch Hand

Joined: Jun 11, 2009
Posts: 47
Hadoop's MapReduce model it is very similar with Google's MapReduce model. If we consider what the creators of the Hadoop tell us, that they started to design Hadoop after reading the GFS and MapReduce papers published by Google, it means that is the same model.

Of course there are some differences. Not so far ago I read Data-Intensive Text processing with MapReduce book written by Jimmy Lin and Chris Dyer, in that book I often saw mentioned the small differences between Hadoop and Google's MapReduce implementation.

I am asking the author of Hadoop in Action, if that book also covers such comparisons or mentions differences on the mode Google's MapReduce implementation was designed compared to Hadoop?
Kuladip Yadav
Ranch Hand

Joined: Jul 30, 2008
Posts: 162

Thanks Tibi for your reply.

But can anybody campare both of these model?

Thanks.
Chuck Lam
author
Greenhorn

Joined: Aug 09, 2010
Posts: 12
Hadoop is based on a couple research papers published by Google explaining Google's data processing model. So the *conceptual model* can be considered the same. Of course, the details are very different.

To start out, Google's MapReduce programs are generally written in C/C++, while Hadoop's are generally Java-based. Given that both models have evolved separately over the years to target different communities, it shouldn't be surprising that the details are very different. Having said that, Google engineers I've talked to claim that learning Hadoop is relatively easy for them.
Tibi Kiss
Ranch Hand

Joined: Jun 11, 2009
Posts: 47
Chuck Lam wrote:....To start out, Google's MapReduce programs are generally written in C/C++, while Hadoop's are generally Java-based. ....

I was expecting that Google also using java implementation.
Thanks for the information!
Kuladip Yadav
Ranch Hand

Joined: Jul 30, 2008
Posts: 162

Thanks Chuck for your information
 
It is sorta covered in the JavaRanch Style Guide.
 
subject: MapReduce model used in Hadoop & google
 
Similar Threads
Hadoop and distributed transactional cache
MongoDB vs Hadoop
How closely coupled is Mahout to Hadoop and MapReduce?
Mapreduce using Java
candidate for map reduce