jQuery in Action, 2nd edition*
The moose likes Hadoop and the fly likes Hadoop - One Map and many Reduces Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of Android Security Essentials Live Lessons this week in the Android forum!
JavaRanch » Java Forums » Databases » Hadoop
Bookmark "Hadoop - One Map and many Reduces" Watch "Hadoop - One Map and many Reduces" New topic
Author

Hadoop - One Map and many Reduces

Luan Cestari
Ranch Hand

Joined: Feb 07, 2010
Posts: 162

Hi Chuck Lam,

Suppose I have some data and I want process it iteratively grouping for a different key. I think this could be done by running some Hadoop Tasks, but each would have an initial load, that is the initial I/O and the mapping process.
My idea was a map once and then do several reduces. Those reduces would emit new maps for the next reduces.
Is it possible to do with Hadoop? What do you think about this approach?

Is there any planning to change the hadoop to run many reduces from the same map? ( I guess then way it is now it can't)

IMO, it look like that if you can do many reduces taking advantage of it is already in the memory would make the process faster than do many jobs with the same map.


Please, visit me for some cool tech post at www.ourdailycodes.com
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: Hadoop - One Map and many Reduces
 
Similar Threads
Hadoop usage examples
Hadoop and compression
Question about using Multi-threading
Beyond Hello World
candidate for map reduce