Andrew Mcmurray

Hi all

Kinda to new to hadoop and running map reduce jobs. I have successfully inserted files into hdfs, ran a map reduce job, and stored the results in an output directory on hdfs. My questions is now that the results are their what is the best way to retrieve them for display? I saw I can connect to the FSFileSystem. Do I do that get a handle to the output dir and then loop through the results? Am I way off or is there better ways? I am looking to either have a Grails app or maybe just a jsp display the results.

Thanks!

Hi Tim,

So I am trying to run it locally using an Eclipse run config. For Example I pull down the config next to the debug configs and run it. I can't figure out how to run multiple apps though. Do you know how to run multiple that way?

thanks,

Amd

Hi all,

Is there a way to run multiple web apps in a signal Eclipse run config?

thanks

AMD

Thanks guys! Bear, I will check out your Front Man impl. I need something without all the bells and whistles bc I am not sure how accepting my new project will be of me if I start demanding we need to implement a new framework The Web app is not too too big, so I think I can kill two birds with one stone. Make the jsps more scriptless and move at least the new code to more of a Model 2.

Thanks,

AMD

Hi all

I read Bear's article "Scriptless JSP Pages: The Front Man" and found it very informative. I pretty much understood what was going on, but just wanted to make sure this makes sense using to you guys. So basically in a Model 2 all requests come into the front controller and then the controller delegates out to certain command classes based on a configuration. I was a little confused as to where this config is kept? I was thinking I could do something like this. Let me know if this sounds correct?

Say I had a jsp and you could either view, update, delete, or create something. Could I just set a request parameter called method send the request to the front controller, have a switch on the method parameter, and then create whatever command class I needed to delegate out to based on the method name? For example if I passed ?method=delete id=1 then the front controller could created a DeleteCommand class do the deletion and then redirect to the view controller which would then fwd back to the jsp? Does this make sense?

Thanks,

AMD

Awesome thanks for the info Bear

Hi Bear,

Thanks for the input. So when you say a business layer are you suggesting a set of classes that the servlet will delegate out to, to handle business logic based off the input entered?

Thanks,

AMD

Hi all

Been out of Web dev for about the last 5 years and lately have been doing all straight Java. Recently I switched projects and inherited a web app down is jsp and servlets that the customer wants customized to their needs. I have a general some design practice questions and looking for guidance before I jump straight in. Basically right now there is a jsp with a form and when the form is submitted certain parties are emailed. The email code and who to send it to right now is in the servlet. The customer wants this to be changed based on a "project" drop down list. For example if project "A" is selected then send an email to the person who submitted the form, a but if project "B" is selected from the drop down maybe send an email to the submitter and maybe the submitter's boss and if project "C" is selected blah blah blah. I am wondering where is the best place to put this kind of logic? Should it go in some JS in the jsp to set some hidden vars and then let the server side figure it out? Should I do all the logic on the server side based off the value in the selected drop down (the project name)? But what if the project name changes??? I want to remain generic flexible enough to be able to handle new project conditions and possible name changes. Any thoughts?

Thanks!

AMD

Thanks Ernest

I will give your second idea a shot. For the 1st idea I do have the paths to the files, but I am dealing with 100K records per file and on average ten files(chunks) so I think the overhead of opening and closing the stream might be a performance killer.

Thanks,

AMD

Hi Ernest,

So basically what I am doing is a merge sort. I have separate file chunks and I need to "peek" at the first record(object) of each file and then return the smallest one. I only want to advance the file pointer in file where the smallest record was found. Does this help?

Thanks,

AMD

when I call the mark supported method it returns false. I need to use ObjectInputStream because I am writing and reading in serialized objects.

Hi all

I want to be able to "peek" at my ObjectInputStream. I thought mark / reset would of worked, but that is not supported with ObjectInputStream. I need to use ObjectInputStream because I am reading and writing serialized objects. Any thoughts on how to accomplish this?

Thanks,

AMD

can you explain this little more detail

I am guessing you want to save order information. Like product name, price, date ordered? You need to know what type of information you want saved before you can implement a solution.

Hi all

I am pretty new to the HDFS and was looking for some opinions on some conflicting answers I have recently gotten.

1. Is it a good idea to compress the stream to write the file out to hadoop. One person told me they had got 10x benefit from doing this. Another told me that it was bad to compress b\c the map reduces that ran on the file could not be distributed using compressed files.

2. I read that map reduces running on hadoop works best with file sizes between 500gb and tb size files. Someone told me that the it works better with smaller files.

Any thoughts?

Thanks,

AMD

Hi all

I am reading from a socketChannel into a ByteBuffer. I want to cut the input by new lines. What is the best way to handle this? Should I flip the buffer convert to a string and then do the split, or is there a why I can do it at the byte level?

Thanks,

AMD