Win a copy of Mesos in Action this week in the Cloud/Virtualizaton forum!
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic

What is the best way to access final result?

 
Nakataa Kokuyo
Ranch Hand
Posts: 189
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Good day,

After MapReduce algorithm completed, should the final result store in text file or what is the proper way to store it?

if i need to access the final result for reporting purposes, what is the best way to access it ?

Thanks in advance and appreciated your enlightenment!
 
Mark Spritzler
ranger
Sheriff
Posts: 17278
6
IntelliJ IDE Mac Spring
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Nakataa Kokuyo wrote:Good day,

After MapReduce algorithm completed, should the final result store in text file or what is the proper way to store it?

if i need to access the final result for reporting purposes, what is the best way to access it ?

Thanks in advance and appreciated your enlightenment!


It depends on what you want to do with it. If you need to give it back to some user from say a search on a website, then why would you save it to a file?

It all depends on your use case.

Mark
 
Nakataa Kokuyo
Ranch Hand
Posts: 189
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi Mark,

I need the final result data for reporting and display it into chart.

What is the approach to store final result ? should it be text file or Hbase?
 
Mark Spritzler
ranger
Sheriff
Posts: 17278
6
IntelliJ IDE Mac Spring
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Nakataa Kokuyo wrote:Hi Mark,

I need the final result data for reporting and display it into chart.

What is the approach to store final result ? should it be text file or Hbase?


So wouldn't you just bring that data back to the caller to create the chart? Why does it have to be saved. Is it something that is later on someone is looking for the chart? Should the chart already be created? Is the chart itself a file? You can store the data anywhere, there isn't one best. It is just what is easiest and when is it needed?

Meaning if I was doing crunching of data from a request in a web page, I would be querying data that is already somewhere, and I just need that data now, in memory so I can generate the chart and send it back to the request.

Hope that helps.

Mark
 
Nakataa Kokuyo
Ranch Hand
Posts: 189
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hey Mark,
The output from MapReduce program doesn't give final, it required a data massage before hand over to a page with chart
 
Srinivas Mupparapu
Greenhorn
Posts: 14
Java
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hello Nakataa,
In an MR program you need to specify the output format for your "reduce" program and final output depends on it. The final result from the reduce program is written to the output folder (in HDFS) that you have specified on your MR Job specification. Number of output files from the reduce program equals the number of reducers that you have specified on your Job Config object.

If you are running HBase MR with the TableOutputFormat as your destiniation then your result is in the HBas table that you specified when you created the job.

Include your code in this post from your MR Driver class if you need further help.
 
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic