aspose file tools*
The moose likes Hadoop and the fly likes What is the best way to access final result? Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Databases » Hadoop
Bookmark "What is the best way to access final result?" Watch "What is the best way to access final result?" New topic
Author

What is the best way to access final result?

Nakataa Kokuyo
Ranch Hand

Joined: Jul 24, 2011
Posts: 189
Good day,

After MapReduce algorithm completed, should the final result store in text file or what is the proper way to store it?

if i need to access the final result for reporting purposes, what is the best way to access it ?

Thanks in advance and appreciated your enlightenment!
Mark Spritzler
ranger
Sheriff

Joined: Feb 05, 2001
Posts: 17257
    
    6

Nakataa Kokuyo wrote:Good day,

After MapReduce algorithm completed, should the final result store in text file or what is the proper way to store it?

if i need to access the final result for reporting purposes, what is the best way to access it ?

Thanks in advance and appreciated your enlightenment!


It depends on what you want to do with it. If you need to give it back to some user from say a search on a website, then why would you save it to a file?

It all depends on your use case.

Mark


Perfect World Programming, LLC - Two Laptop Bag - Tube Organizer
How to Ask Questions the Smart Way FAQ
Nakataa Kokuyo
Ranch Hand

Joined: Jul 24, 2011
Posts: 189
Hi Mark,

I need the final result data for reporting and display it into chart.

What is the approach to store final result ? should it be text file or Hbase?
Mark Spritzler
ranger
Sheriff

Joined: Feb 05, 2001
Posts: 17257
    
    6

Nakataa Kokuyo wrote:Hi Mark,

I need the final result data for reporting and display it into chart.

What is the approach to store final result ? should it be text file or Hbase?


So wouldn't you just bring that data back to the caller to create the chart? Why does it have to be saved. Is it something that is later on someone is looking for the chart? Should the chart already be created? Is the chart itself a file? You can store the data anywhere, there isn't one best. It is just what is easiest and when is it needed?

Meaning if I was doing crunching of data from a request in a web page, I would be querying data that is already somewhere, and I just need that data now, in memory so I can generate the chart and send it back to the request.

Hope that helps.

Mark
Nakataa Kokuyo
Ranch Hand

Joined: Jul 24, 2011
Posts: 189
Hey Mark,
The output from MapReduce program doesn't give final, it required a data massage before hand over to a page with chart
Srinivas Mupparapu
Greenhorn

Joined: Feb 12, 2004
Posts: 14

Hello Nakataa,
In an MR program you need to specify the output format for your "reduce" program and final output depends on it. The final result from the reduce program is written to the output folder (in HDFS) that you have specified on your MR Job specification. Number of output files from the reduce program equals the number of reducers that you have specified on your Job Config object.

If you are running HBase MR with the TableOutputFormat as your destiniation then your result is in the HBas table that you specified when you created the job.

Include your code in this post from your MR Driver class if you need further help.
 
 
subject: What is the best way to access final result?