aspose file tools
The moose likes Hadoop and the fly likes Hadoop MapReduce Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login

Win a copy of Badass: Making Users Awesome this week in the Game Development forum!
JavaRanch » Java Forums » Databases » Hadoop
Bookmark "Hadoop MapReduce" Watch "Hadoop MapReduce" New topic

Hadoop MapReduce

Ralph Hoch

Joined: Jun 04, 2011
Posts: 4

I'm new to Hadoop and I'm trying to figure out how it works. As for an exercise I should implement something similar to the WordCount-Example. The task is to read in several files, do the WordCount and write and output file for each input file.
Hadoop uses a combiner and shuffles the output of the map-part as an input for the reducer. Then writes one output file (I guess for each instance that is running). I was wondering if it is possible to write one output file for each input file (so keep the words of inputfile1 and write result to outputfile1 and so on). Is it possible to overwrite the Combiner-Class or is there another solution for this (I'm not sure if this should even be solved in a Hadoop-Task but this is the exercise).

Satyaprakash Joshii
Ranch Hand

Joined: Jun 18, 2012
Posts: 140
You need to process each file separately........and number of reducers should not be more than 1 to ensure 1 o\p file for each input file...
Gartner says :Bigdata will be most advanced analytics products by 2015 !

Time to Become Big data architect by learning Hadoop(Developer, Administration,Analyst,QA),Cassandra,MongoDb,HBase,Datascience, Mahout, Splunk,R etc) from scratch to expert level
subject: Hadoop MapReduce