Kinda to new to hadoop and running map reduce jobs. I have successfully inserted files into hdfs, ran a map reduce job, and stored the results in an output directory on hdfs. My questions is now that the results are their what is the best way to retrieve them for display? I saw I can connect to the FSFileSystem. Do I do that get a handle to the output dir and then loop through the results? Am I way off or is there better ways? I am looking to either have a Grails app or maybe just a jsp display the results.
There are coulple of options depending on your file format:
- As you mentioned, you can use FSFileSystem in a Servlet to read and process the data and display it in your JSP.
- You could create a Hive external table and use HiveQL with JDBC to retrieve your data into your JSP. Keep in mind that HiveQL is similar to SQL but with limited features.