Need to search large log files present in a remote unix server

[Sivapraveen Chandana]

Hi all,
I need to search large log files present in a remote unix server. I know that I can use SFTP APIs like Jcraft's jsch api to connect to the remote server but then copying those log files to my local machine would not be a good idea since each of them is at least 500 MB in size.
I need to search through 5 or 6 such log files based on the search string input by the user and then copy relevant search data and display it to the user.
Since, the log files are too big in size, I am thinking of the possibility of running a shell script in the remote server that can do the task of searching and displaying relevant results, through my java program.
As of now, this process is done manually through the Putty client and I need to automate it.
Is there a better way of achieving what I need to do?

Please advise.
Thanks in advance..

[Martijn Verburg]

Hmm, interesting problem. Have yo considered looking at open source search engines that can search through text. For example Lucene?

Jsch IIRC also gives you ways of interacting through an SSH session, so in theory you could open up a session and execute a script.

Hope that helps a little!